tools: Add example LAVA pull lab script - #1561
Conversation
| def poll_loop(args): | ||
| global timestamp |
There was a problem hiding this comment.
What happens if a user runs the script once, quits out and later runs the script again?
pollevents() uses this timestamp as a "from" marker when looking for new jobs.
Do we want to initialise timestamp here to be the time that the script is run? Or store the "latest" used timestamp to a file?
There was a problem hiding this comment.
I will redesign it with persistence, for timestamp
| f"Processing job {node_id} (name: {node.get('name')}, " | ||
| f"platform: {platform}, runtime: {runtime})" | ||
| ) | ||
| processed.add(node_id) |
There was a problem hiding this comment.
I guess processed will just keep getting larger and larger forever? (until the script is quit and re-run)
There was a problem hiding this comment.
Good point, i will adjust it and add expiration and persistence for the processed nodes.
| def poll_loop(args): | ||
| global timestamp | ||
|
|
||
| processed = set() |
There was a problem hiding this comment.
Should we be saving the "processed" jobs to a file for subsequent script runs to build upon?
|
FYI I've created #1562 to use to test this PR with |
Labs running LAVA behind a firewall cannot use the push-style LAVA integration as the pipeline has no URL to reach them. When a lava runtime is configured without a url (e.g. lava-testpull), the pipeline stores the rendered LAVA job definition in external storage and publishes its URL as the job_definition node artifact instead. Add tools/example_pull_lab_lava.py which polls the events API for such jobs, filtered by a mandatory lab name and optional platforms and device types, and submits the definitions verbatim to a local LAVA instance via its REST API. Results are reported by LAVA itself through the notify callback embedded in the definition, so the lab only needs outbound connectivity. Submitted jobs and the polling position are persisted in an append-only JSONL journal that tolerates interrupted writes and is compacted automatically, so restarting the script does not resubmit jobs or replay old events. Also document the LAVA pull lab flow in doc/connecting-pull-lab.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
31cbb9c to
ac622f9
Compare
2 participants
Labs running LAVA behind a firewall cannot use the push-style LAVA integration as the pipeline has no URL to reach them. When a lava runtime is configured without a url (e.g. lava-testpull), the pipeline stores the rendered LAVA job definition in external storage and publishes its URL as the job_definition node artifact instead.
Add tools/example_pull_lab_lava.py which polls the events API for such jobs, filtered by a mandatory lab name and optional platforms and device types, and submits the definitions verbatim to a local LAVA instance via its REST API. Results are reported by LAVA itself through the notify callback embedded in the definition, so the lab only needs outbound connectivity.
Also document the LAVA pull lab flow in doc/connecting-pull-lab.md.
Assisted-By: Claude Fable 5 noreply@anthropic.com