Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions contributing/BACKENDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,23 @@ git clone https://github.com/dstackai/gpuhunt.git
- **Online providers** offer dynamic machine configurations that are available at the very moment
when you fetch configurations (e.g., GPU marketplaces).
`gpuhunt` collects online providers' instance offers each time a `dstack` user provisions a new instance.
Examples: `tensordock`, `vastai`, etc.
Examples: `vastai`, `hotaisle`, etc.

### 1.3. Create the provider class

Create the provider class file under `src/gpuhunt/providers`.

Make sure your class extends the [`AbstractProvider`](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/__init__.py)
base class. See its docstrings for descriptions of the methods that your class should implement.
Make sure your class extends either `OnlineProvider` or `OfflineProvider` from
[base.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/base.py),
matching the choice you made above.

Both kinds implement `get`, returning `CatalogItem`s with `provider` set to your provider's `NAME`.
Additionally:

- Online providers implement the `from_env` classmethod, which reads credentials from the
environment with `get_creds_env` and raises `MissingCredsError` if one is missing. Providers that
raise it are skipped by `default_catalog()` rather than failing the whole catalog.
- Offline providers may override `filter` to omit some offers from the published catalog.

Refer to examples:
- Offline providers:
Expand All @@ -47,8 +56,8 @@ Refer to examples:
[azure.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/azure.py),
[lambdalabs.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/lambdalabs.py).
- Online providers:
[vultr.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/vultr.py)
[tensordock.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/tensordock.py),
[vultr.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/vultr.py),
[hotaisle.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/hotaisle.py),
[vastai.py](https://github.com/dstackai/gpuhunt/blob/main/src/gpuhunt/providers/vastai.py).

### 1.4. Register the provider with the catalog
Expand All @@ -57,7 +66,8 @@ Add your provider in the following places:
- Either `OFFLINE_PROVIDERS` or `ONLINE_PROVIDERS` in `src/gpuhunt/_internal/catalog.py`.
- The `python -m gpuhunt` command in `src/gpuhunt/__main__.py`.
- (offline providers) The CI workflow in `.github/workflows/catalogs.yml`.
- (online providers) The default catalog in `src/gpuhunt/_internal/default.py`.
- (online providers) `ONLINE_PROVIDER_MODULES` in `src/gpuhunt/_internal/default.py`, which is what
`default_catalog()` loads.

### 1.5. Add data quality tests

Expand Down
20 changes: 17 additions & 3 deletions contributing/GPUHUNT.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,27 @@

An offer is a possible configuration. It consists of:
- Provider (or backend in dstack)
- CPU architecture
- CPU count
- RAM size
- Disk size
- GPU count
- GPU vendor (if any)
- GPU model name (if any)
- GPU VRAM size (if any)
- Is interruptible (or spot)
- Region (provider-specific)
- Instance name or ID (provider-specific)
- Price per hour

Offers are represented by `CatalogItem`. Providers construct them directly, setting `provider` to
their own `NAME`, and `gpu_vendor` whenever `gpu_count` is non-zero.

## Catalog

Some providers don't have a suitable API for querying all offers in real-time. That's why gpuhunt has two types of providers:

- Online — offers can be queried in real-time
- Online — offers can be queried quickly in real-time
- Offline — offers must be loaded from a precomputed catalog file

The `Catalog` class hides those details from the user, reading offers from the file for offline providers or querying online providers.
Expand All @@ -28,7 +33,16 @@ The `Catalog` class pulls the latest catalog from the S3 bucket and caches it fo

## Provider implementation

Providers must implement a single method `get`. It has the same name for both online and offline providers but works differently.
Providers subclass either `OnlineProvider` or `OfflineProvider` from `src/gpuhunt/providers/base.py`.

Both implement `get`. It has the same name for both online and offline providers but works differently.

In addition:

- Online providers implement the `from_env` classmethod, since `default_catalog()` constructs them
in the user's process.
- Offline providers may override `filter` to omit some offers from the published catalog.
Credentials are passed in by the caller, so a missing one is an error rather than a skip.

### Offers sorting

Expand Down Expand Up @@ -100,7 +114,7 @@ These mechanisms are used to preserve backward compatibility:

- **`gpuhunt` version**: The interfaces in the `gpuhunt` package preserve backward compatibility
within a minor version (`X` in `0.X.Y`).
- **Offer flags**: If an offer breaks older `dstack` versions, it is marked with a flag in `RawCatalogItem.flags`
- **Offer flags**: If an offer breaks older `dstack` versions, it is marked with a flag in `CatalogItem.flags`
and the flag is added to the list of supported flags in `dstack`.
Older `dstack` versions that don't support this flag will not see the respective offers.
- **Offline catalog versions**: If a breaking change in the structure or content of an offline catalog is unavoidable,
Expand Down
Loading