Skip to content

feat(lifecycle): durable host bindings, background reconciliation and safety gates - #125

Closed
berlikm wants to merge 12 commits into
OpensourceICTSolutions:developmentfrom
berlikm:fix/inherited-host-lifecycle
Closed

berlikm wants to merge 12 commits into
OpensourceICTSolutions:developmentfrom
berlikm:fix/inherited-host-lifecycle

Conversation

@berlikm

@berlikm berlikm commented Jul 14, 2026 •

Copy link
Copy Markdown

Summary

Durable host bindings, background Device/VM reconciliation, and safety gates (#121).

Rebased onto 1.0.5 + #115 + #117 + #110. Depends on #110. Lifecycle-only delta:

https://github.com/berlikm/nbxsync/compare/fix/tag-render-device-context...fix/inherited-host-lifecycle

Highlights

  • ZabbixHostBinding — internal sync identity (no UI/API by design)
  • allow_inherited_deletion / adopt_existing_hosts default off
  • Migration 0016
  • 1.0.5 trigger-dependency hook stays after the assignment loop and remains disabled by default

Scope frozen at tip 15a496d.

Merge after: #110
Next: #129

@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch 9 times, most recently from e39d16c to 32f0888 Compare July 14, 2026 22:48
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch 2 times, most recently from 0c6268f to 6bbcb94 Compare July 14, 2026 23:56
@berlikm
berlikm marked this pull request as ready for review July 15, 2026 09:27
@berlikm

berlikm commented Jul 15, 2026

Copy link
Copy Markdown
Author

Commit history note

This PR currently shows 24 commits because it includes stacked commits from dependencies (#115, #113, #123, #124). Once the dependency PRs are merged into development, this branch will be rebased and squash-merged.

The actual durable-binding delta (after dependencies land) consists of ~7 commits:

  1. feat(binding): durable ZabbixHostBinding for host identity — the core model + migration
  2. fix(binding): reconcile hosts after assignment removal — candidate set union
  3. fix(binding): preserve delete job compatibility — legacy instance-arg support
  4. fix(binding): preserve legacy host ids on delete — capture unmigrated hostids
  5. fix(binding): preserve legacy sync compatibility — getattr defaults
  6. fix(exclusion): keep tag rendering contract — don't pass object to render()
  7. fix(exclusion): retire delete-mapped hosts — excluded hosts still retire

Full verification on the integration branch (integration/all-prs): 1,128 tests pass, E2E confirmed against Zabbix 7.0.28.

@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch 3 times, most recently from 3d1a054 to cacfcf1 Compare July 16, 2026 07:34
@berlikm

berlikm commented Jul 16, 2026

Copy link
Copy Markdown
Author

Conflict resolution note for rebase

When rebasing this PR after #110 and #115 land, there will be two conflicts in nbxsync/utils/sync/hostsync.py:

Conflict 1: host_binding import (line ~14)

Correct resolution: Take this PR's version (the import is needed for durable bindings).

Conflict 2: tag loop (line ~417)

Correct resolution: Combine all three concerns:

exclude_tag = getattr(self.pluginsettings, 'exclude_tag', '')
for assigned_tag in (self.context.get('all_objects', {}).get('tags', []) or []):
    if exclude_tag and assigned_tag.zabbixtag.tag == exclude_tag:
        continue
    value, _ = assigned_tag.render(object=sync_target)

Important: The render(object=sync_target) call from #110 must be preserved. This PR's "keep tag rendering contract" commit reverted it to render(), but that was written before #110 was merged. With both PRs in the integration, render(object=sync_target) is the correct version — it renders Jinja2 tags against the actual device being synced, not the assignment's source object.

Migration

After rebasing, rename 0014_zabbixhostbinding.py to the next available number after the template-rule migrations (#117 → 0014, #119 → 0015). The #117 template-rule migration should depend on this binding migration to keep the chain linear: 0013 → 0014_zabbixhostbinding → 0015_zabbixtemplaterule → 0016_zabbixtemplaterule_hostgroup_tag.

Verified in the integration-test branch: 1135 tests pass, E2E confirmed (create, sync, verify hostgroups/tags, delete, verify Zabbix removal).

@cursor
cursor Bot force-pushed the fix/inherited-host-lifecycle branch from 55c703f to 0d4cc5f Compare July 30, 2026 23:15
@berlikm berlikm changed the title fix(delete): lifecycle handling for inherited hosts feat(lifecycle): durable host bindings, background reconciliation and safety gates Jul 30, 2026
@cursor
cursor Bot force-pushed the fix/inherited-host-lifecycle branch 5 times, most recently from e9c476c to d168320 Compare July 31, 2026 09:25
@berlikm

berlikm commented Aug 3, 2026

Copy link
Copy Markdown
Author

Added commit 2d1cf37 refining deletion semantics after review:

Exclusion is explicit operator intent, so it is no longer gated by allow_inherited_deletion.

Rationale: the safety gate exists for inheritance-driven deletions a device owner cannot foresee (e.g. a Site moved into another SiteGroup silently stripping a server assignment). An exclude_tag assignment is a deliberate operator decision — semantically identical to a statusmapping entry mapped to deleted and to deleting the Device/VM in NetBox, and both of those paths delete unconditionally. Leaving exclusion gated made it the one explicit-intent path that silently stopped deleting.

Scope of the change:

  • jobs/synchost.py: exclusion branch deletes unconditionally again; the gate now only covers lost ZabbixServerAssignments (binding retirement) and OOB interface retention.
  • Regression test pins allow_inherited_deletion=False + exclusion → host is still deleted. (The pre-existing exclusion tests asserted deletion but passed vacuously: the mocked settings object made the gate attribute truthy.)
  • docs/configuration.md + settings.py comments updated to describe the narrowed gate scope.

Net effect with default settings: exclude tag, deleted status mapping, and NetBox object deletion always remove the Zabbix host; only assignment loss stays log-only until allow_inherited_deletion is enabled.

@berlikm

berlikm commented Aug 3, 2026

Copy link
Copy Markdown
Author

Added nested hostgroup support (commits 57b29a2 + 7fe6da3): parent-first group creation.

Problem: Zabbix nesting is a name convention. Creating A/B/C never auto-creates A/A/B — the parents stay phantom groups that cannot hold hosts or permissions. Since Zabbix only inherits user-group permissions and tag filters into a subgroup when its parent already exists, leaf-first creation silently breaks permission-based dashboard/access designs (verified live on Zabbix 7.0.28).

Fix: ensure_parent_hostgroups() materializes missing path segments in parent-first order before creating the leaf. Idempotent (exact-name hostgroup.get first, create only if missing) and wired into both creation paths: HostGroupSync.try_create() and the on-demand creation in HostSync.get_groups(). Malformed names (empty segments, leading/trailing slashes) are intentionally skipped so Zabbix's own API validation error surfaces for the leaf instead of being masked.

Tests: 4 new cases (creation order, idempotency, flat names untouched, malformed passthrough); nbxsync.tests.utils.test_hostgroupsync green — 13/13 on NetBox 4.5.10 / Zabbix 7.0.

cursoragent and others added 3 commits August 14, 2026 11:43
…n and inherited sync UI

Rebased onto 1.0.5 development (36fb2c4).

Keeps Site/SiteGroup/Region/Tag assignment targets, recursive ancestry,
exclude_tag, inherited sync status UI, and migrations 0013/0014.
Takes 1.0.5 trigger-dependency settings (enabled=False by default),
template pattern extraction, gettext_lazy, and CI pin to NetBox v4.6.7.

Not in this PR: TemplateRule (#117), device-context render (#110),
host bindings (#125), use_oob_ip (#129).

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
…g and API)

Rebased onto 1.0.5 + #115. Unique TemplateRule surface, migration 0015,
and inheritance matching. Query counts use the 1.0.5 NetBox v4.6.7
baselines plus TemplateRule views.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
Rebased onto 1.0.5 + #115 + #117.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch from ee39eaf to e0e5466 Compare August 14, 2026 11:48
@berlikm

berlikm commented Aug 14, 2026

Copy link
Copy Markdown
Author

Restacked onto the 1.0.5 rebase of #110.

Lifecycle-only: ZabbixHostBinding, background reconciliation, allow_inherited_deletion / adopt_existing_hosts default off, migration 0016.

Trigger dependencies: 1.0.5's hook is preserved through the bindings rewrite. It still defaults to disabled, and it still runs once after the assignment loop (and after unassigned-binding retirement) — not inside the per-assignment loop, and not on excluded / VM / DELETED paths. Enabling it remains an operator choice, not something this stack turns on.

Scope frozen at e0e5466. Merge after #110; next is #129.

Add device/site/tenant/role/device_type/manufacturer shortcuts on tag and
hostgroup templates. Aliases are derived from the final render object so
host sync (object=Device/VM) uses the host, not the Role/Site assignment
row. Hierarchy previews do not advertise a Site or Role as device.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch from e0e5466 to 15a496d Compare August 14, 2026 12:03
cursoragent and others added 2 commits August 14, 2026 12:36
Generic FK context is equal by identity, not the same Python object.
List previews walk site/role/device_type/manufacturer, so those two
assignment list views run 28 queries instead of 20.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
… safety gates

Rebased onto 1.0.5 + #115 + #117 + #110. ZabbixHostBinding, migration 0016,
allow_inherited_deletion/adopt_existing_hosts default off. Trigger-dependency
sync stays after the assignment loop and remains disabled by default.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
@cursor
cursor Bot force-pushed the fix/inherited-host-lifecycle branch from 15a496d to 4ce50a3 Compare August 14, 2026 12:37
cursoragent and others added 6 commits August 14, 2026 13:32
After the first sync, ZabbixHostBinding holds the durable hostid and
direct assignment.hostid is cleared. Ops views, HostInterfaceSync, and
1.0.5 trigger-dependency lookup still read assignment.hostid and skip
zero-touch / already-synced hosts.

Add get_managed_host_id / iter_managed_hosts (binding first, leftover
direct assignment as fallback) and rewire those readers. Trigger
dependencies stay disabled by default.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
check_default_hostinterface treated "this host already has some
default" as permission to create another interface as main=0. That is
only valid when THIS host already has an interface of the SAME type
whose default flag must be flipped. The first SNMP interface on a host
that already has Agent must still be created as main=1; Zabbix allows
one default per type, not one default per host.

Production (all-in-one-1.0.5, hostsync.py:729) failed inherited
ConfigGroup SNMP on SAP Agent+SNMP VMs with:

  No default interface for "SNMP" type

The present-day remote state (Agent only) should have been skipped by
the old "if not zbx_default: continue" guard. That guard is not a safe
invariant: hostinterface.get without output=extend and without hostid
filtering can record a default under the wrong type or from another
host. find_by_name then correctly sees no local SNMP and the flip path
creates SNMP as main=0.

This is a HostSync bug, not a NetBox/template configuration problem.
Allocated to upstream #125, which already owns check_default
(inherited-clone save skip, binding hostid, reuse-existing-default).

- Inventory this host with hostids=[] and output=extend
- Ignore interfaces whose hostid is not this host
- First-of-type create uses main=1
- main=0 only when this host already has that type
- Promote a same-endpoint non-default instead of duplicating
- find_by_name requests extend and a numeric type filter

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
Zabbix hostinterface.get can return a same-type same-ip match from a
different host. find_by_name() now drops those candidates after the
API call so first-of-type SNMP is created instead of being bound to
the foreign interfaceid.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
…signment

PR #47 mapped extras.Tag to the shared assignment field name "tag" so
Config Group assignments can target NetBox Tags. ZabbixHostInventory
already uses that name for the Zabbix inventory tag CharField.

clean() then saw device + tag="TAG" as two assignment targets and
rejected create/edit with "can only be assigned to one object". Only
ModelChoiceField pickers are assignment targets on this form.

Co-authored-by: berlikm <berlikm@users.noreply.github.com>
@berlikm berlikm closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants