OCW OER Export

This demonstration project showcases how to utilize the MIT Learn API. It specifically focuses on extracting MIT OpenCourseWare courses' metadata and creating a CSV file for export to OER Commons, aligning with their specific requirements.

SECTIONS

1. Initial Setup & Usage
2. Requirements
3. Local Development
4. Tests
5. Committing & Formatting

Initial Setup & Usage

1. Build the container:

docker compose build

2. Start the container:

docker compose run --rm app

To generate a JSON file containing complete API data:

docker compose run --rm app --create_json

To create a CSV file from the local JSON file:

docker compose run --rm app --create_csv --source=json

File Output Directory

The output files, whether in CSV or JSON format, are stored within the private/output directory relative to the current working directory from which the command is executed.

Therefore, the above commands will generate private/output/ocw_oer_export.csv or private/output/ocw_api_data.json in the current working directory.

If you want to change this, you will not only have to change the output_path in the function (create_csv or create_json) but also have to change the mapping in docker-compose.yml.

Environment Variables

By default, this project uses MIT Learn's Production API, as given in ocw_oer_export/config.py. To use the RC API or local, create an environment file, .env in the project's root directory and add the relevant base URL: e.g., API_BASE_URL=https://api.rc.learn.mit.edu or API_BASE_URL=http://localhost:8063

Requirements

For successful execution and correct output, ensure the MIT Learn's API contains the following fields:

title, url, runs: level, description, topics, runs: instructors, runs: semester, runs: year, course_feature Additionally, the mapping_files should be up-to-date. If new topics are added in OCW without corresponding mappings in ocw_oer_export/mapping_files/ocw_topic_to_oer_subject.csv, this will lead to null entries for those topics in the CSV (CR_SUBJECT). In addition to that, make sure fm_keywords_exports.csv is also present.

Local Development

This project uses uv to manage Python and its dependencies. To set up a local environment and run the CLI without Docker:

uv sync
uv run python -m ocw_oer_export.cli --create_csv

Tests

To run unit tests:

docker run --rm ocw_oer_export python -m unittest discover -s tests

Or, locally:

uv run python -m unittest discover -s tests

Committing & Formatting

To ensure commits to GitHub are safe, install the git hook with prek, which runs the hooks in .pre-commit-config.yaml (prek is pinned in the project's dependencies):

uv sync
uv run prek install -f

-f replaces any git hook that pre-commit installed earlier.

Running prek can confirm your commit is safe to be pushed to GitHub and correctly formatted:

uv run prek run --all-files