Feature/digitalocean source

README.md

@@ -5,6 +5,7 @@ This is a tool to sync data from different sources to a NetBox instance.
 Available source types:
 * VMware vCenter Server
 * [bb-ricardo/check_redfish](https://github.com/bb-Ricardo/check_redfish) inventory files
+* [DigitalOcean](docs/source_digitalocean.md)
 
 **IMPORTANT: READ INSTRUCTIONS CAREFULLY BEFORE RUNNING THIS PROGRAM**
 

docs/source_digitalocean.md

@@ -0,0 +1,116 @@
+# DigitalOcean source
+
+The `digitalocean` source reads Droplets, attached block volumes, and managed
+database clusters from the DigitalOcean API and reconciles them into NetBox.
+API access is read-only; inventory changes are written to NetBox by the normal
+netbox-sync process.
+
+## Installation and access
+
+The source uses the `requests` dependency already installed by netbox-sync; no
+DigitalOcean SDK is required. Create a DigitalOcean API token with read access
+to Droplets and volumes. If managed database ingestion is enabled, the token
+must also be able to read databases.
+
+Keep the API token out of version control and provide it through a protected
+configuration file or the application's standard source environment variables.
+
+## Configuration
+
+Add a source section to the existing settings file:
+
+```ini
+[source/example-digitalocean]
+type = digitalocean
+api_token = REPLACE_WITH_READ_ONLY_API_TOKEN
+region_mapping = fra1=Germany, nyc1=United States
+sync_volumes = True
+sync_databases = True
+```
+
+`api_token` is required. `sync_volumes` and `sync_databases` default to `True`.
+Set either to `False` when the token should not query that API resource.
+Accounts without managed databases can leave `sync_databases` enabled:
+both `{"databases": []}` and `{"databases": null}` are treated as empty
+database collections. Missing collection fields and other malformed values
+remain errors rather than being treated as empty inventory.
+Generate the complete configuration with:
+
+```sh
+python netbox-sync.py -g
+```
+
+`region_mapping` is optional. It accepts comma-separated, exact
+`DigitalOcean region slug=NetBox Region name` pairs. Quote destination names
+containing commas, for example `nyc1="New York, USA"`. Duplicate source slugs,
+empty entries, and Region names longer than 100 characters are rejected.
+
+On NetBox 4.2+, the source reuses a Region with the destination name, then
+tries its generated slug, and creates it if neither exists. Explicit mappings
+take precedence over Regions matching the DigitalOcean region. Without a
+mapping, the DigitalOcean region name and slug are used (for example
+`Frankfurt 1` and `fra1`). When only a region slug is provided by DigitalOcean,
+it is also used as the name. Existing Regions are not renamed.
+
+Multiple DigitalOcean regions can map to the same NetBox Region; their
+virtualization clusters remain separate and retain their DigitalOcean names.
+Existing clusters in the source's cluster group are updated in place when
+their scope changes, including previously unscoped clusters. Ambiguous
+duplicate clusters in that group cause an explicit error instead of choosing
+one arbitrarily.
+
+The NetBox token needs permission to add Regions when they do not already
+exist. On NetBox versions before 4.2, Region creation and mapping are not
+applied; clusters remain unscoped.
+
+## NetBox mapping
+
+| DigitalOcean | NetBox |
+| --- | --- |
+| Region | Virtualization cluster scoped to a reused or automatically created NetBox Region on NetBox 4.2+ |
+| Droplet | Virtual machine |
+| Droplet size | VM vCPUs and memory |
+| Droplet image distribution/name | VM platform |
+| Droplet public/private IPv4 and IPv6 | VM interfaces and IP addresses |
+| Droplet root disk | VM disk capacity on older NetBox; virtual disk on NetBox 3.7+ |
+| Attached block volume | Additional virtual disk on NetBox 3.7+ |
+| DigitalOcean resource tags | NetBox tags |
+| Managed database cluster | VM inventory record with `db-` name prefix and metadata custom fields |
+
+DigitalOcean reports Droplet and volume disk sizes in gigabytes; these are
+converted to megabytes for NetBox using 1024 MB per reported GB. Database
+clusters are represented as one VM-like record per cluster because this
+project has no native managed-database object type. Database engine, version,
+size, node count, endpoint host, and port are stored in `digitalocean_*` custom
+fields. Connection credentials and connection URIs are never copied. Database
+endpoint hostnames are not resolved into IP addresses.
+
+DigitalOcean private and public network addresses are assigned to interfaces
+named after their network type. The source does not create NetBox prefixes or
+sites.
+
+Droplets with the same name in different clusters remain distinct. Interfaces
+and virtual disks are matched to their resolved parent VM, not just its name.
+If an earlier run moved child objects between same-named VMs, inspect the next
+dry run for interface, disk, and primary-IP changes before enabling writes.
+
+IP assignments can be repaired automatically when the existing IP belongs to
+the current source, either through this run's ownership or its persisted
+`Source: <source name>` tag. Conflicting source ownership or uncertain ownership
+stops the run before the normal NetBox write/prune phase instead of silently
+skipping a discovered IP. Two interfaces claiming the same IP in one source run
+also cause an explicit error. Source section names must remain stable for
+persisted ownership tags to match. If an earlier run already removed an IP's
+source tag or marked it orphaned, verify its owner and restore the correct tag
+or repair its assignment manually before retrying.
+
+## Validation
+
+Start with a dry run:
+
+```sh
+python netbox-sync.py -n -c settings.ini
+```
+
+Confirm the discovered Droplet, volume, and database cluster counts and inspect
+the resulting NetBox object changes before enabling writes or pruning.

module/config/file_output.py

@@ -17,6 +17,7 @@
 from module.sources.vmware.config import VMWareConfig
 from module.sources.check_redfish.config import CheckRedfishConfig
 from module.sources.hetzner.config import HetznerConfig
+from module.sources.digitalocean.config import DigitalOceanConfig
 from module.common.logging import get_logger
 from module.config import default_config_file_path, source_config_section_name
 from module.config.files import ConfigFile, ConfigFileINI, ConfigFileYAML
@@ -35,7 +36,8 @@ class ConfigFileOutput(DescriptionFormatterMixin):
     source_config_list = [
         VMWareConfig,
         CheckRedfishConfig,
-        HetznerConfig
+        HetznerConfig,
+        DigitalOceanConfig
     ]
 
     header = f"Welcome to the {__description__} configuration file."

module/netbox/inventory.py

@@ -128,6 +128,15 @@ def get_by_data(self, object_type, data=None):
 
         # try to find by primary/secondary key
         elif data.get(object_type.primary_key) is not None:
+            if object_type in (NBVMInterface, NBVirtualDisk) and isinstance(data.get("virtual_machine"), NBVM):
+                # VM names are only unique within a cluster; use the resolved parent object.
+                for this_object in self.get_all_items(object_type):
+                    if this_object.data.get("virtual_machine") is data["virtual_machine"] and \
+                            str(this_object.data.get(object_type.primary_key)).lower() == \
+                            str(data[object_type.primary_key]).lower():
+                        return this_object
+                return None
+
             object_name_to_find = None
             for this_object in self.get_all_items(object_type):
 

module/netbox/object_classes.py

@@ -1470,7 +1470,6 @@ class NBRegion(NetBoxObject):
     object_type = "dcim.region"
     primary_key = "name"
     prune = False
-    read_only = True
 
     def __init__(self, *args, **kwargs):
         self.data_model = {

module/sources/__init__.py

@@ -10,6 +10,7 @@
 # define all available sources here
 from module.sources.vmware.connection import VMWareHandler
 from module.sources.hetzner.connection import HetznerHandler
+from module.sources.digitalocean.connection import DigitalOceanHandler
 from module.sources.check_redfish.import_inventory import CheckRedfish
 
 from module.common.logging import get_logger
@@ -19,7 +20,7 @@
 from module.config import source_config_section_name
 
 # list of valid sources
-valid_sources = [VMWareHandler, CheckRedfish, HetznerHandler]
+valid_sources = [VMWareHandler, CheckRedfish, HetznerHandler, DigitalOceanHandler]
 
 
 def validate_source(source_class_object=None, state="pre"):

module/sources/digitalocean/__init__.py

@@ -0,0 +1,8 @@
+# -*- coding: utf-8 -*-
+#  Copyright (c) 2020 - 2026 netbox-sync team. All rights reserved.
+#
+#  This work is licensed under the terms of the MIT license.
+#  For a copy, see file LICENSE.txt included in this
+#  repository or visit: <https://opensource.org/licenses/MIT>.
+
+from module.sources.digitalocean.connection import DigitalOceanHandler

module/sources/digitalocean/client.py

@@ -0,0 +1,72 @@
+# -*- coding: utf-8 -*-
+#  Copyright (c) 2020 - 2026 netbox-sync team. All rights reserved.
+#
+#  This work is licensed under the terms of the MIT license.
+#  For a copy, see file LICENSE.txt included in this
+#  repository or visit: <https://opensource.org/licenses/MIT>.
+
+from urllib.parse import urljoin, urlparse
+
+import requests
+
+
+class DigitalOceanClient:
+
+    api_url = "https://api.digitalocean.com/v2/"
+    timeout = 30
+
+    def __init__(self, token, session=None):
+        self.session = session or requests.Session()
+        self.headers = {
+            "Accept": "application/json",
+            "Authorization": f"Bearer {token}",
+            "User-Agent": "netbox-sync",
+        }
+
+    def get_collection(self, path, collection_name):
+        url = urljoin(self.api_url, path.lstrip("/"))
+        params = {"per_page": 200}
+        objects = []
+
+        while url is not None:
+            response = self.session.get(
+                url,
+                headers=self.headers,
+                params=params,
+                timeout=self.timeout,
+            )
+            response.raise_for_status()
+            payload = response.json()
+            if not isinstance(payload, dict):
+                raise ValueError(f"DigitalOcean response for '{path}' is not a JSON object.")
+            collection = payload.get(collection_name)
+            # DigitalOcean can return {"databases": null} for accounts without databases.
+            if collection_name == "databases" and "databases" in payload and collection is None:
+                collection = []
+            if not isinstance(collection, list):
+                raise ValueError(f"DigitalOcean response for '{path}' has no '{collection_name}' list.")
+            objects.extend(collection)
+
+            links = payload.get("links") or {}
+            if not isinstance(links, dict):
+                raise ValueError(f"DigitalOcean response for '{path}' has invalid pagination links.")
+            pages = links.get("pages") or {}
+            if not isinstance(pages, dict):
+                raise ValueError(f"DigitalOcean response for '{path}' has invalid pagination pages.")
+            next_url = pages.get("next")
+            if next_url:
+                if not isinstance(next_url, str):
+                    raise ValueError(f"DigitalOcean response for '{path}' has a non-string pagination link.")
+                url = urljoin(url, next_url)
+                parsed_url = urlparse(url)
+                if parsed_url.scheme != "https" or parsed_url.hostname != "api.digitalocean.com" \
+                        or parsed_url.port not in (None, 443) or parsed_url.username is not None:
+                    raise ValueError("DigitalOcean API returned a pagination link outside its HTTPS API host.")
+            else:
+                url = None
+            params = None
+
+        return objects
+
+    def close(self):
+        self.session.close()

module/sources/digitalocean/config.py

@@ -0,0 +1,87 @@
+# -*- coding: utf-8 -*-
+#  Copyright (c) 2020 - 2026 netbox-sync team. All rights reserved.
+#
+#  This work is licensed under the terms of the MIT license.
+#  For a copy, see file LICENSE.txt included in this
+#  repository or visit: <https://opensource.org/licenses/MIT>.
+
+from module.common.logging import get_logger
+from module.common.misc import quoted_split
+from module.config import source_config_section_name
+from module.config.base import ConfigBase
+from module.config.option import ConfigOption
+
+log = get_logger()
+
+
+class DigitalOceanConfig(ConfigBase):
+
+    section_name = source_config_section_name
+    source_name_example = "my-digitalocean-example"
+
+    def __init__(self):
+        self.options = [
+            ConfigOption(
+                "enabled",
+                bool,
+                default_value=True,
+                description="Enable or disable the DigitalOcean source."
+            ),
+            ConfigOption(
+                "type",
+                str,
+                default_value="digitalocean",
+                description="Source type identifier. Must remain 'digitalocean'."
+            ),
+            ConfigOption(
+                "api_token",
+                str,
+                mandatory=True,
+                sensitive=True,
+                description="DigitalOcean API token with read access to the resources being synced."
+            ),
+            ConfigOption(
+                "region_mapping",
+                str,
+                config_example="fra1=Germany, nyc1=United States",
+                description="Optional comma-separated DigitalOcean region slug=NetBox Region name mappings. "
+                            "Missing Regions are created on NetBox 4.2+. Unmapped regions use the "
+                            "DigitalOcean region name and slug."
+            ),
+            ConfigOption(
+                "sync_volumes",
+                bool,
+                default_value=True,
+                description="Sync attached block volumes as virtual disks."
+            ),
+            ConfigOption(
+                "sync_databases",
+                bool,
+                default_value=True,
+                description="Sync managed database clusters as VM inventory records."
+            ),
+        ]
+
+        super().__init__()
+
+    def validate_options(self):
+        option = self.get_option_by_name("region_mapping")
+        mapping = {}
+        if option.value is not None:
+            if not isinstance(option.value, str):
+                log.error("Config option 'region_mapping' must be a comma-separated string of slug=name mappings.")
+                self.set_validation_failed()
+                return
+
+            for entry in quoted_split(option.value):
+                slug, separator, name = entry.partition("=")
+                slug = slug.strip(' "\'')
+                name = name.strip(' "\'')
+                if not separator or not slug or not name or len(name) > 100 or slug in mapping:
+                    log.error(f"Invalid region_mapping entry '{entry}': expected a unique region slug and "
+                              "a non-empty NetBox Region name of at most 100 characters (slug=name).")
+                    self.set_validation_failed()
+                    continue
+                mapping[slug] = name
+
+        option.set_value(mapping)