StaticFetcher() Implementation + from_pdb() refactor

[jauy123]

Fixes #5429 and #5431

Changes made in this Pull Request:

• Implements fetch.fetchers.StaticFetcher()
• Refactors fetch.pdb.from_pdb() to use StaticFetchers()

LLM / AI generated code disclosure

LLMs or other AI-powered tools (beyond simple IDE use cases) were used in this contribution: yes / no
From comment: #5436 (comment)

Per AI policy, I'm declaring that I used in AI in primary two ways: IDE autocomplete and help with reviewing the docstring used for the documentation to make it look more professional.

The one notable exception is that I used AI to help me draft test_multiple_downloads_existing_database and test_existing_database in testsuite/MDAnalysisTests/fetch/test_static_fetcher.py

I prompt is "Is there a unique way to determine the hash in order if two files are exactly the same including modification time in python using the standard library"

It gave code that used os.stat(), so I googled it and found pathlib.Path.stat().st_mtime in the official [docs](https://docs.python.org/3/library/pathlib.html#pathlib.Path.stat) which I then used manually for test_existing_database and test_multiple_downloads_existing_database with no AI.

PR Checklist

• Issue raised/referenced?
• Tests updated/added?
• Documentation updated/added?
• package/CHANGELOG file updated?
• Is your name in package/AUTHORS? (If it is not, add it!)
• LLM/AI disclosure was updated.

Developers Certificate of Origin

I certify that I can submit this code contribution as described in the Developer Certificate of Origin, under the MDAnalysis LICENSE.

[Copilot]

Pull request overview

This PR introduces a new MDAnalysis.fetch.fetchers.StaticFetcher abstraction for downloading/caching files via pooch, and refactors MDAnalysis.fetch.pdb.from_PDB() to use this new fetcher as its backend.

Changes:

• Adds StaticFetcher implementation with registry read/write/append helpers.
• Refactors from_PDB() to delegate downloads to StaticFetcher.
• Adds/updates tests and Sphinx module docs for the new fetcher module.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
testsuite/MDAnalysisTests/fetch/test_static_fetcher.py	New test coverage for StaticFetcher behavior and error cases.
testsuite/MDAnalysisTests/fetch/test_from_PDB.py	Updates from_PDB() tests for the refactored implementation.
testsuite/MDAnalysisTests/fetch/servers.py	Adds a local HTTP server helper for fetcher tests.
package/MDAnalysis/fetch/pdb.py	Refactors from_PDB() to use StaticFetcher and renames supported-format constant.
package/MDAnalysis/fetch/fetchers.py	Adds the StaticFetcher implementation and supporting utilities/constants.
package/MDAnalysis/fetch/init.py	Exposes StaticFetcher via package imports (but needs __all__ alignment).
package/doc/sphinx/source/documentation_pages/fetchers/fetchers.rst	Adds Sphinx automodule page for MDAnalysis.fetch.fetchers.
package/doc/sphinx/source/documentation_pages/fetchers_modules.rst	Registers the new fetchers documentation page in the module index.
.gitignore	Adds ignores for local scripts (should be reverted/clarified).

Comments suppressed due to low confidence (2)

package/MDAnalysis/fetch/init.py:41

• __all__ does not include StaticFetcher, but the module imports it, so from MDAnalysis.fetch import * will not export the new public symbol consistently.

__all__ = ["from_PDB"]

from .pdb import from_PDB
from .fetchers import StaticFetcher

package/MDAnalysis/fetch/fetchers.py:633

• In _set_downloader(), when downloader="auto" and base_url doesn't match the regex (or is missing a scheme), _downloader is never set, leading to UnboundLocalError. Also, there is no default case for unsupported schemes, so the function can silently return None.

        # Regex matching if downloader is set to auto
        if downloader == "auto":
            regex = r"^([^:]+):"
            match = re.match(regex, base_url)

            if match:
                _downloader = match.group(1)

        else:
            _downloader = downloader

        match _downloader:
            case "http" | "https":
                return pooch.HTTPDownloader(**kwargs)
            case "ftp":
                return pooch.FTPDownloader(**kwargs)
            case "sftp":
                return pooch.SFTPDownloader(**kwargs)
            case "doi":
                return pooch.DOIDownloader(**kwargs)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jauy123]

This is currently intentional. Should it be changed?

[jauy123]

Sleeping on it, I think the best behavior would have like an argument that making it so that the function appends to the registry unconditionally.

The default behavior should be the registry is not appended if there is a duplicate.

[orbeckst]

Can you use

from .servers import temporary_http_server

?

[jauy123]

This breaks testsuite/MDAnalysisTests/utils/test_imports.py

[jauy123]

I have no idea what trigger copilot on this. I didn't turn on anything

[orbeckst]

@jauy123 , the copilot reviews are currently enabled to automatically occur on all PRs. I am not sure if that was intentional or just a setting changed on GitHub. If you (or anyone else) wants to jump into a discussion, I posted on discord #developers https://discord.com/channels/807348386012987462/808088023957897258/1530342254256979978

[Copilot]

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (3)

testsuite/MDAnalysisTests/fetch/test_static_fetcher.py:28

• from servers import ... is likely to fail under pytest because the test root is on sys.path, not this subdirectory, so servers.py may not be importable as the top-level module name servers. Import it via the MDAnalysisTests package namespace to make the test import robust.

from servers import temporary_http_server

testsuite/MDAnalysisTests/fetch/servers.py:55

• The test HTTP server binds to a fixed port (7123). This can cause spurious failures when the port is already in use (or when tests run in parallel). Binding to port 0 lets the OS pick a free ephemeral port.

        server = ThreadingHTTPServer(("127.0.0.1", 7123), http_handler)

package/MDAnalysis/fetch/pdb.py:163

• from_PDB(..., progressbar=...) forwards progressbar into StaticFetcher.fetch() as an unknown kwarg. StaticFetcher.fetch() currently forwards unknown kwargs into the downloader constructor, which will likely raise a TypeError (e.g., HTTPDownloader(..., progressbar=...)). Pass this through the existing verbose parameter instead.

    return fetcher.fetch(
        file_name=_pdb_ids,
        base_url="https://files.wwpdb.org/download/",
        progressbar=progressbar,
    )

[orbeckst]

fair comment!

[jauy123]

Indeed, a fair comment.

However, by design, this fixture exists to test the StaticFetcher().fetch(cache_path=None) argument which is guarded by requiring pooch to run the corresponding test. All the other tests uses the tmp_path fixture instead.

So the NameError should never run in this case.

[jauy123]

To make it more obvious, I can add a guard saying that pooch is required for clean_up_default_cache()