Skip to content

KubeRay API Reference

KubeRay integration components for running Ray on Kubernetes. Learn how to use it here.

Authentication

For connecting to auth-protected Ray clusters, see Authentication docs. Client-mode resources support Ray's token authentication via environment variables. Pipes clients (PipesKubeRayJobClient) additionally accept headers, cookies, and verify parameters for HTTP-level authentication.

Custom host resolution

Override BaseKubeRayResource.resolve_hostname to customize how the Ray head service address is constructed (e.g. name.namespace.company.com instead of the default in-cluster FQDN).


Client Mode Resources

These resources initialize Ray client connection with a remote cluster.

dagster_ray.kuberay.KubeRayInteractiveJob pydantic-model

Bases: BaseKubeRayResource

Provides a Ray Job for Dagster steps.

Is the recommended way to run Ray workloads with automatic cluster management. It creates a Ray Job, connects to it in client mode and sets the jobId field. Cleanup is handled by the KubeRay controller or by the resource lifecycle logic.

Info

Image defaults to dagster/image run tag.

Tip

Make sure ray[full] is available in the image.

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "ExecutionOptionsConfig": {
      "properties": {
        "cpu": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Cpu"
        },
        "gpu": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gpu"
        },
        "object_store_memory": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Object Store Memory"
        }
      },
      "title": "ExecutionOptionsConfig",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "InteractiveRayJobConfig": {
      "description": "Same as [`RayJobConfig`][dagster_ray.kuberay.configs.RayJobConfig], but `spec.submission_mode` mode has to be `InteractiveMode`",
      "properties": {
        "kind": {
          "default": "RayJob",
          "title": "Kind",
          "type": "string"
        },
        "api_version": {
          "default": "ray.io/v1",
          "title": "Api Version",
          "type": "string"
        },
        "metadata": {
          "additionalProperties": true,
          "description": "Kubernetes metadata, except the name field can be omitted. In this case it will be generated by `dagster-ray`.",
          "title": "Metadata",
          "type": "object"
        },
        "spec": {
          "$ref": "#/$defs/InteractiveRayJobSpec"
        }
      },
      "title": "InteractiveRayJobConfig",
      "type": "object"
    },
    "InteractiveRayJobSpec": {
      "additionalProperties": true,
      "description": "Same as [`RayJobSpec`][dagster_ray.kuberay.configs.RayJobSpec], but `mode` has to be `InteractiveMode`",
      "properties": {
        "active_deadline_seconds": {
          "default": 86400,
          "title": "Active Deadline Seconds",
          "type": "integer"
        },
        "pre_running_deadline_seconds": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Deadline for the `RayJob` to reach the `Running` state, measured from `.status.startTime`. If it doesn't, KubeRay fails the job with reason `PreRunningDeadlineExceeded`. Useful for reaping jobs stuck in `Initializing` or `Waiting` \u2014 for example when the `RayCluster` can never be scheduled. Unset means no deadline. Requires KubeRay 1.6.0: older operators prune the field without an error.",
          "title": "Pre Running Deadline Seconds"
        },
        "backoff_limit": {
          "default": 0,
          "title": "Backoff Limit",
          "type": "integer"
        },
        "ray_cluster_spec": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterSpec"
            },
            {
              "type": "null"
            }
          ]
        },
        "submitter_pod_template": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Submitter Pod Template"
        },
        "submitter_config": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Submitter Config"
        },
        "metadata": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Metadata"
        },
        "cluster_selector": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Cluster Selector"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "deletion_strategy": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "`RayJob` cleanup policy. Unset by default: cleanup is governed by `shutdown_after_job_finishes`, which deletes the `RayCluster` once the job succeeds or fails. Requires KubeRay 1.5.0 (the field was named `deletionPolicy` before that) **and** the `RayJobDeletionPolicy` feature gate \u2014 the KubeRay controller fails the `RayJob` with `ValidationFailed` if the gate is off. The gate is alpha in 1.5.x (off by default) and beta in 1.6.x (on by default). See [Deletion Strategy](../tutorial/kuberay.md#deletion-strategy).",
          "title": "Deletion Strategy"
        },
        "runtime_env_yaml": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Runtime Env Yaml"
        },
        "job_id": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Job Id"
        },
        "submission_mode": {
          "const": "InteractiveMode",
          "default": "InteractiveMode",
          "title": "Submission Mode",
          "type": "string"
        },
        "entrypoint_resources": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Resources"
        },
        "entrypoint_num_cpus": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Num Cpus"
        },
        "entrypoint_memory": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Memory"
        },
        "entrypoint_num_gpus": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Num Gpus"
        },
        "ttl_seconds_after_finished": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": 300,
          "title": "Ttl Seconds After Finished"
        },
        "shutdown_after_job_finishes": {
          "default": true,
          "title": "Shutdown After Job Finishes",
          "type": "boolean"
        },
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        }
      },
      "title": "InteractiveRayJobSpec",
      "type": "object"
    },
    "Lifecycle": {
      "properties": {
        "create": {
          "default": true,
          "description": "Whether to create the resource. If set to `False`, the user can manually call `.create` instead.",
          "title": "Create",
          "type": "boolean"
        },
        "wait": {
          "default": true,
          "description": "Whether to wait for the remote Ray cluster to become ready to accept connections. If set to `False`, the user can manually call `.wait` instead.",
          "title": "Wait",
          "type": "boolean"
        },
        "connect": {
          "default": true,
          "description": "Whether to run `ray.init` against the remote Ray cluster. If set to `False`, the user can manually call `.connect` instead.",
          "title": "Connect",
          "type": "boolean"
        },
        "cleanup": {
          "default": "always",
          "description": "Resource cleanup policy. Determines when the resource should be deleted after Dagster step execution or during interruption.",
          "enum": [
            "never",
            "always",
            "on_exception"
          ],
          "title": "Cleanup",
          "type": "string"
        }
      },
      "title": "Lifecycle",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "RayDataExecutionOptions": {
      "properties": {
        "execution_options": {
          "$ref": "#/$defs/ExecutionOptionsConfig"
        },
        "cpu_limit": {
          "default": 5000,
          "title": "Cpu Limit",
          "type": "integer"
        },
        "gpu_limit": {
          "default": 0,
          "title": "Gpu Limit",
          "type": "integer"
        },
        "verbose_progress": {
          "default": true,
          "title": "Verbose Progress",
          "type": "boolean"
        },
        "use_polars": {
          "default": true,
          "title": "Use Polars",
          "type": "boolean"
        }
      },
      "title": "RayDataExecutionOptions",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "description": "Provides a Ray Job for Dagster steps.\n\nIs the recommended way to run Ray workloads with automatic cluster management. It creates a Ray Job, connects to it in client mode and sets the `jobId` field. Cleanup is handled by the KubeRay controller or by the resource lifecycle logic.\n\nInfo:\n    Image defaults to `dagster/image` run tag.\n\nTip:\n    Make sure `ray[full]` is available in the image.",
  "properties": {
    "lifecycle": {
      "$ref": "#/$defs/Lifecycle",
      "description": "Actions to perform during resource setup."
    },
    "timeout": {
      "default": 600.0,
      "description": "Timeout for Ray readiness in seconds",
      "title": "Timeout",
      "type": "number"
    },
    "ray_init_options": {
      "additionalProperties": true,
      "description": "Additional keyword arguments to pass to `ray.init()` call, such as `runtime_env`, `num_cpus`, etc. Dagster's `EnvVar` is supported. More details in [Ray docs](https://docs.ray.io/en/latest/ray-core/api/doc/ray.init.html).",
      "title": "Ray Init Options",
      "type": "object"
    },
    "data_execution_options": {
      "$ref": "#/$defs/RayDataExecutionOptions"
    },
    "redis_port": {
      "default": 10001,
      "description": "Redis port for connection. Make sure to match with the actual available port.",
      "title": "Redis Port",
      "type": "integer"
    },
    "dashboard_port": {
      "default": 8265,
      "description": "Dashboard port for connection. Make sure to match with the actual available port.",
      "title": "Dashboard Port",
      "type": "integer"
    },
    "env_vars": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "description": "Environment variables to pass to the Ray cluster.",
      "title": "Env Vars"
    },
    "enable_tracing": {
      "default": false,
      "description": "Enable tracing: inject `RAY_PROFILING=1` and `RAY_task_events_report_interval_ms=0` into the Ray cluster configuration. This allows using `ray.timeline()` to fetch recorded task events. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-core/api/doc/ray.timeline.html#ray-timeline)",
      "title": "Enable Tracing",
      "type": "boolean"
    },
    "enable_actor_task_logging": {
      "default": false,
      "description": "Enable actor task logging: inject `RAY_ENABLE_RECORD_ACTOR_TASK_LOGGING=1` into the Ray cluster configuration.",
      "title": "Enable Actor Task Logging",
      "type": "boolean"
    },
    "enable_debug_post_mortem": {
      "default": false,
      "description": "Enable post-mortem debugging: inject `RAY_DEBUG_POST_MORTEM=1` into the Ray cluster configuration. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-observability/ray-distributed-debugger.html)",
      "title": "Enable Debug Post Mortem",
      "type": "boolean"
    },
    "enable_legacy_debugger": {
      "default": false,
      "description": "Enable legacy debugger: inject `RAY_DEBUG=legacy` into the Ray cluster configuration. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-observability/user-guides/debug-apps/ray-debugging.html#using-the-ray-debugger)",
      "title": "Enable Legacy Debugger",
      "type": "boolean"
    },
    "image": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Image to inject into the `RayCluster` spec. Defaults to `dagster/image` run tag. Images already provided in the `RayCluster` spec won't be overridden.",
      "title": "Image"
    },
    "deployment_name": {
      "default": "dev",
      "description": "Dagster deployment name. Is used as a prefix for the Kubernetes resource name. Dagster Cloud variables are used to determine the default value.",
      "title": "Deployment Name",
      "type": "string"
    },
    "failure_tolerance_timeout": {
      "default": 0.0,
      "description": "The period in seconds to wait for the cluster to transition out of `failed` state if it reaches it. This state can be transient under certain conditions. With the default value of 0, the first `failed` state appearance will raise an exception immediately.",
      "title": "Failure Tolerance Timeout",
      "type": "number"
    },
    "poll_interval": {
      "default": 1.0,
      "description": "Poll interval for various API requests",
      "title": "Poll Interval",
      "type": "number"
    },
    "ray_job": {
      "$ref": "#/$defs/InteractiveRayJobConfig",
      "description": "Configuration for the Kubernetes `RayJob` CR"
    },
    "client": {
      "description": "Kubernetes `RayJob` client",
      "title": "Client"
    },
    "log_cluster_conditions": {
      "default": true,
      "description": "Whether to log `RayCluster` conditions while waiting for the RayCluster to become ready. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/observability.html#raycluster-status-conditions).",
      "title": "Log Cluster Conditions",
      "type": "boolean"
    }
  },
  "title": "KubeRayInteractiveJob",
  "type": "object"
}

Fields:

Attributes

lifecycle pydantic-field
lifecycle: Lifecycle

Actions to perform during resource setup.

ray_job pydantic-field

Configuration for the Kubernetes RayJob CR

client pydantic-field
client: ResourceDependency[RayJobClient]

Kubernetes RayJob client

failure_tolerance_timeout pydantic-field
failure_tolerance_timeout: float = 0.0

The period in seconds to wait for the cluster to transition out of failed state if it reaches it. This state can be transient under certain conditions. With the default value of 0, the first failed state appearance will raise an exception immediately.

log_cluster_conditions pydantic-field
log_cluster_conditions: bool = True

Whether to log RayCluster conditions while waiting for the RayCluster to become ready. Learn more: KubeRay docs.

dagster_ray.kuberay.KubeRayCluster pydantic-model

Bases: BaseKubeRayResource

Provides a Ray Cluster for Dagster steps.

It is advised to use KubeRayInteractiveJob with KubeRay >= 1.3.0 instead.

Info

Image defaults to dagster/image run tag.

Tip

Make sure ray[full] is available in the image.

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "ClusterSharing": {
      "description": "Defines the strategy for sharing `RayCluster` resources with other Dagster steps.\n\nBy default, the cluster is expected to be created by Dagster during one of the previously executed steps.\n\n!!!note\n\n    Cluster sharing uses the Kubernetes Lease API (`coordination.k8s.io`) for leader election\n    to coordinate cluster creation across parallel steps. The Dagster ServiceAccount must have\n    `create`, `get`, and `delete` permissions on `leases` in the `coordination.k8s.io` API group.",
      "properties": {
        "enabled": {
          "default": false,
          "description": "Whether to enable sharing of RayClusters.",
          "title": "Enabled",
          "type": "boolean"
        },
        "match_dagster_labels": {
          "$ref": "#/$defs/MatchDagsterLabels",
          "description": "Configuration for matching on Dagster-generated labels."
        },
        "match_labels": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Additional user-provided labels to match on.",
          "title": "Match Labels"
        },
        "ttl_seconds": {
          "default": 1800.0,
          "description": "Time to live for the lock placed on the `RayCluster` resource, marking it as in use by the current Dagster step. The lock is renewed periodically while the step is running (see `heartbeat`), so this only needs to cover the gap between renewals.",
          "title": "Ttl Seconds",
          "type": "number"
        },
        "heartbeat": {
          "$ref": "#/$defs/ClusterSharingHeartbeat",
          "description": "Configuration for background renewal of the cluster sharing lock while the Dagster step is running."
        }
      },
      "title": "ClusterSharing",
      "type": "object"
    },
    "ClusterSharingHeartbeat": {
      "description": "Controls background renewal of the cluster sharing lock while the Dagster step is running.\n\nWithout renewal, the lock expires `ttl_seconds` after step start and the garbage collection\nsensor may delete the cluster while the step is still using it. With renewal, `ttl_seconds`\ncan stay short: idle clusters are reaped promptly, active steps survive.\n\nEach renewal updates the lock's `heartbeat_at` timestamp; `created_at` always points at the\ninitial lock placement. A hanging step renews its lock indefinitely \u2014 set the\n`dagster/max_runtime` tag on runs to bound step runtime.",
      "properties": {
        "enabled": {
          "default": true,
          "description": "Whether to renew the cluster sharing lock in the background while the Dagster step is running.",
          "title": "Enabled",
          "type": "boolean"
        },
        "refresh_seconds": {
          "default": 10.0,
          "description": "How often to renew the lock. Must be well below `ClusterSharing.ttl_seconds` \u2014 if a renewal is missed, the lock must not have expired yet, or the cluster can be deleted mid-step.",
          "title": "Refresh Seconds",
          "type": "number"
        }
      },
      "title": "ClusterSharingHeartbeat",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "ExecutionOptionsConfig": {
      "properties": {
        "cpu": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Cpu"
        },
        "gpu": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gpu"
        },
        "object_store_memory": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Object Store Memory"
        }
      },
      "title": "ExecutionOptionsConfig",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "Lifecycle": {
      "properties": {
        "create": {
          "default": true,
          "description": "Whether to create the resource. If set to `False`, the user can manually call `.create` instead.",
          "title": "Create",
          "type": "boolean"
        },
        "wait": {
          "default": true,
          "description": "Whether to wait for the remote Ray cluster to become ready to accept connections. If set to `False`, the user can manually call `.wait` instead.",
          "title": "Wait",
          "type": "boolean"
        },
        "connect": {
          "default": true,
          "description": "Whether to run `ray.init` against the remote Ray cluster. If set to `False`, the user can manually call `.connect` instead.",
          "title": "Connect",
          "type": "boolean"
        },
        "cleanup": {
          "default": "always",
          "description": "Resource cleanup policy. Determines when the resource should be deleted after Dagster step execution or during interruption.",
          "enum": [
            "never",
            "always",
            "on_exception"
          ],
          "title": "Cleanup",
          "type": "string"
        }
      },
      "title": "Lifecycle",
      "type": "object"
    },
    "MatchDagsterLabels": {
      "properties": {
        "cluster_sharing": {
          "default": true,
          "description": "Whether to match on `dagster/cluster-sharing=true` label.",
          "title": "Cluster Sharing",
          "type": "boolean"
        },
        "code_location": {
          "default": true,
          "description": "Whether to match on `dagster/code-location` label. The value will be taken from the current Dagster code location.",
          "title": "Code Location",
          "type": "boolean"
        },
        "resource_key": {
          "default": true,
          "description": "Whether to match on `dagster/resource-key` label. The value will be taken from the current Dagster resource key.",
          "title": "Resource Key",
          "type": "boolean"
        },
        "git_sha": {
          "default": true,
          "description": "Whether to match on `dagster/git-sha` label. The value will be taken from `DAGSTER_CLOUD_GIT_SHA` environment variable.",
          "title": "Git Sha",
          "type": "boolean"
        },
        "run_id": {
          "default": false,
          "description": "Whether to match on `dagster/run-id` label. The value will be taken from the current Dagster run ID.",
          "title": "Run Id",
          "type": "boolean"
        }
      },
      "title": "MatchDagsterLabels",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterConfig": {
      "properties": {
        "kind": {
          "default": "RayCluster",
          "title": "Kind",
          "type": "string"
        },
        "api_version": {
          "default": "ray.io/v1",
          "title": "Api Version",
          "type": "string"
        },
        "metadata": {
          "additionalProperties": true,
          "description": "Kubernetes metadata, except the name field can be omitted. In this case it will be generated by `dagster-ray`.",
          "title": "Metadata",
          "type": "object"
        },
        "spec": {
          "$ref": "#/$defs/RayClusterSpec"
        }
      },
      "title": "RayClusterConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "RayDataExecutionOptions": {
      "properties": {
        "execution_options": {
          "$ref": "#/$defs/ExecutionOptionsConfig"
        },
        "cpu_limit": {
          "default": 5000,
          "title": "Cpu Limit",
          "type": "integer"
        },
        "gpu_limit": {
          "default": 0,
          "title": "Gpu Limit",
          "type": "integer"
        },
        "verbose_progress": {
          "default": true,
          "title": "Verbose Progress",
          "type": "boolean"
        },
        "use_polars": {
          "default": true,
          "title": "Use Polars",
          "type": "boolean"
        }
      },
      "title": "RayDataExecutionOptions",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "description": "Provides a Ray Cluster for Dagster steps.\n\nIt is advised to use [`KubeRayInteractiveJob`][dagster_ray.kuberay.resources.KubeRayInteractiveJob] with KubeRay >= 1.3.0 instead.\n\nInfo:\n    Image defaults to `dagster/image` run tag.\n\nTip:\n    Make sure `ray[full]` is available in the image.",
  "properties": {
    "lifecycle": {
      "$ref": "#/$defs/Lifecycle",
      "description": "Actions to perform during resource setup."
    },
    "timeout": {
      "default": 600.0,
      "description": "Timeout for Ray readiness in seconds",
      "title": "Timeout",
      "type": "number"
    },
    "ray_init_options": {
      "additionalProperties": true,
      "description": "Additional keyword arguments to pass to `ray.init()` call, such as `runtime_env`, `num_cpus`, etc. Dagster's `EnvVar` is supported. More details in [Ray docs](https://docs.ray.io/en/latest/ray-core/api/doc/ray.init.html).",
      "title": "Ray Init Options",
      "type": "object"
    },
    "data_execution_options": {
      "$ref": "#/$defs/RayDataExecutionOptions"
    },
    "redis_port": {
      "default": 10001,
      "description": "Redis port for connection. Make sure to match with the actual available port.",
      "title": "Redis Port",
      "type": "integer"
    },
    "dashboard_port": {
      "default": 8265,
      "description": "Dashboard port for connection. Make sure to match with the actual available port.",
      "title": "Dashboard Port",
      "type": "integer"
    },
    "env_vars": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "description": "Environment variables to pass to the Ray cluster.",
      "title": "Env Vars"
    },
    "enable_tracing": {
      "default": false,
      "description": "Enable tracing: inject `RAY_PROFILING=1` and `RAY_task_events_report_interval_ms=0` into the Ray cluster configuration. This allows using `ray.timeline()` to fetch recorded task events. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-core/api/doc/ray.timeline.html#ray-timeline)",
      "title": "Enable Tracing",
      "type": "boolean"
    },
    "enable_actor_task_logging": {
      "default": false,
      "description": "Enable actor task logging: inject `RAY_ENABLE_RECORD_ACTOR_TASK_LOGGING=1` into the Ray cluster configuration.",
      "title": "Enable Actor Task Logging",
      "type": "boolean"
    },
    "enable_debug_post_mortem": {
      "default": false,
      "description": "Enable post-mortem debugging: inject `RAY_DEBUG_POST_MORTEM=1` into the Ray cluster configuration. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-observability/ray-distributed-debugger.html)",
      "title": "Enable Debug Post Mortem",
      "type": "boolean"
    },
    "enable_legacy_debugger": {
      "default": false,
      "description": "Enable legacy debugger: inject `RAY_DEBUG=legacy` into the Ray cluster configuration. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-observability/user-guides/debug-apps/ray-debugging.html#using-the-ray-debugger)",
      "title": "Enable Legacy Debugger",
      "type": "boolean"
    },
    "image": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Image to inject into the `RayCluster` spec. Defaults to `dagster/image` run tag. Images already provided in the `RayCluster` spec won't be overridden.",
      "title": "Image"
    },
    "deployment_name": {
      "default": "dev",
      "description": "Dagster deployment name. Is used as a prefix for the Kubernetes resource name. Dagster Cloud variables are used to determine the default value.",
      "title": "Deployment Name",
      "type": "string"
    },
    "failure_tolerance_timeout": {
      "default": 0.0,
      "description": "The period in seconds to wait for the cluster to transition out of `failed` state if it reaches it. This state can be transient under certain conditions. With the default value of 0, the first `failed` state appearance will raise an exception immediately.",
      "title": "Failure Tolerance Timeout",
      "type": "number"
    },
    "poll_interval": {
      "default": 1.0,
      "description": "Poll interval for various API requests",
      "title": "Poll Interval",
      "type": "number"
    },
    "cluster_sharing": {
      "$ref": "#/$defs/ClusterSharing",
      "description": "Configuration for sharing the `RayCluster` across Dagster steps. Existing clusters matching this configuration will be reused without recreating them. A `dagster/sharing=true` label will be applied to the `RayCluster`, and a `dagster/lock-<run-id>-<step-id>=<lock>` annotation will be placed on the `RayCluster` to mark it as being used by this step. Cleanup will only proceed if the `RayCluster` is not being used by any other steps, therefore cluster sharing should be used in conjunction with [dagster_ray.kuberay.sensors.cleanup_expired_kuberay_clusters][] sensor."
    },
    "ray_cluster": {
      "$ref": "#/$defs/RayClusterConfig",
      "description": "Kubernetes `RayCluster` CR configuration."
    },
    "client": {
      "description": "Kubernetes `RayCluster` client",
      "title": "Client"
    },
    "log_cluster_conditions": {
      "default": true,
      "description": "Whether to log RayCluster conditions while waiting for the RayCluster to become ready. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/observability.html#raycluster-status-conditions).",
      "title": "Log Cluster Conditions",
      "type": "boolean"
    }
  },
  "title": "KubeRayCluster",
  "type": "object"
}

Fields:

Attributes

cluster_sharing pydantic-field
cluster_sharing: ClusterSharing

Configuration for sharing the RayCluster across Dagster steps. Existing clusters matching this configuration will be reused without recreating them. A dagster/sharing=true label will be applied to the RayCluster, and a dagster/lock-<run-id>-<step-id>=<lock> annotation will be placed on the RayCluster to mark it as being used by this step. Cleanup will only proceed if the RayCluster is not being used by any other steps, therefore cluster sharing should be used in conjunction with dagster_ray.kuberay.sensors.cleanup_expired_kuberay_clusters sensor.

lifecycle pydantic-field
lifecycle: Lifecycle

Actions to perform during resource setup.

ray_cluster pydantic-field
ray_cluster: RayClusterConfig

Kubernetes RayCluster CR configuration.

client pydantic-field
client: ResourceDependency[RayClusterClient]

Kubernetes RayCluster client

failure_tolerance_timeout pydantic-field
failure_tolerance_timeout: float = 0.0

The period in seconds to wait for the cluster to transition out of failed state if it reaches it. This state can be transient under certain conditions. With the default value of 0, the first failed state appearance will raise an exception immediately.

log_cluster_conditions pydantic-field
log_cluster_conditions: bool = True

Whether to log RayCluster conditions while waiting for the RayCluster to become ready. Learn more: KubeRay docs.


Pipes

dagster_ray.kuberay.PipesKubeRayJobClient

PipesKubeRayJobClient(
    client: RayJobClient | None = None,
    context_injector: PipesContextInjector | None = None,
    message_reader: PipesMessageReader | None = None,
    forward_termination: bool = True,
    timeout: float = 600,
    poll_interval: float = 1,
    port_forward: bool = False,
    address: str | None = None,
    headers: dict[str, Any] | None = None,
    verify: str | bool = True,
    cookies: dict[str, Any] | None = None,
    metadata: dict[str, Any] | None = None,
)

Bases: PipesClient, TreatAsResourceParam

A pipes client for running RayJob on Kubernetes.

Most constructor arguments can be overridden in the run method.

Parameters:

  • context_injector (PipesContextInjector | None, default: None ) –

    A context injector to use to inject context into the RayJob. Defaults to PipesEnvContextInjector.

  • message_reader (PipesMessageReader | None, default: None ) –

    A message reader to use to read messages from the glue job run. Defaults to PipesRayJobMessageReader.

  • client (RayJobClient | None, default: None ) –

    The Kubernetes API client.

  • forward_termination (bool, default: True ) –

    Whether to terminate the Ray job when the Dagster process receives a termination signal, or if the startup timeout is reached. Defaults to True.

  • timeout (float, default: 600 ) –

    Timeout for various internal interactions with the Kubernetes RayJob.

  • poll_interval (float, default: 1 ) –

    Interval at which to poll Kubernetes for status updates.

  • port_forward (bool, default: False ) –

    Whether to use Kubernetes port-forwarding to connect to the KubeRay cluster. Is useful when running in a local environment.

  • address (str | None, default: None ) –

    Ray dashboard address. Passed to JobSubmissionClient. Generated by resolve_hostname if not provided.

  • headers (dict[str, Any] | None, default: None ) –

    HTTP headers for Ray Dashboard requests. Passed to JobSubmissionClient.

  • verify (str | bool, default: True ) –

    Whether to verify TLS certificate. Passed to JobSubmissionClient.

  • cookies (dict[str, Any] | None, default: None ) –

    HTTP cookies for Ray Dashboard requests. Passed to JobSubmissionClient.

  • metadata (dict[str, Any] | None, default: None ) –

    Ray Job metadata. Passed to JobSubmissionClient.

Info

Image defaults to dagster/image run tag.

Tip

Make sure ray[full] is available in the image.

Source code in src/dagster_ray/kuberay/pipes.py
def __init__(
    self,
    client: RayJobClient | None = None,
    context_injector: PipesContextInjector | None = None,
    message_reader: PipesMessageReader | None = None,
    forward_termination: bool = True,
    timeout: float = 600,
    poll_interval: float = 1,
    port_forward: bool = False,
    address: str | None = None,
    headers: dict[str, Any] | None = None,
    verify: str | bool = True,
    cookies: dict[str, Any] | None = None,
    metadata: dict[str, Any] | None = None,
):
    self.client: RayJobClient = client or RayJobClient()

    self._context_injector = context_injector or PipesEnvContextInjector()

    self._message_reader = message_reader or PipesRayJobMessageReader(
        job_submission_client_kwargs={
            "headers": headers,
            "verify": verify,
            "cookies": cookies,
            "metadata": metadata,
        }
    )

    self.forward_termination = check.bool_param(forward_termination, "forward_termination")
    self.timeout = check.numeric_param(timeout, "timeout")
    self.poll_interval = check.numeric_param(poll_interval, "poll_interval")
    self.port_forward = check.bool_param(port_forward, "port_forward")
    self.address = address
    self.headers = headers
    self.verify = verify
    self.cookies = cookies
    self.metadata = metadata

    self._job_submission_client: JobSubmissionClient | None = None

Functions

run
run(
    *,
    context: OpOrAssetExecutionContext,
    submit_job_params: SubmitJobParams | None = None,
    ray_job: dict[str, Any] | None = None,
    extras: PipesExtras | None = None,
    address: str | None = None,
    headers: dict[str, Any] | None = None,
    verify: str | bool | None = None,
    cookies: dict[str, Any] | None = None,
    metadata: dict[str, Any] | None = None
) -> PipesClientCompletedInvocation

Execute a RayJob, enriched with the Pipes protocol.

Parameters passed to JobSubmissionClient take precedence over the constructor arguments.

At least one of submit_job_params or ray_job must be provided.

Parameters:

  • context (OpOrAssetExecutionContext) –

    Current Dagster op or asset context.

  • submit_job_params (SubmitJobParams | None, default: None ) –

    Job submission parameters (entrypoint, runtime_env, etc.). When combined with ray_job, values are merged into the template. When used alone, a minimal manifest is auto-generated using the dagster/image run tag.

  • ray_job (dict[str, Any] | None, default: None ) –

    KubeRay RayJob template dict. Can be used alone or together with submit_job_params.

  • extras (PipesExtras | None, default: None ) –

    Additional information to pass to the Pipes session, retrievable via PipesContext.get_extras.

  • address (str | None, default: None ) –

    Ray dashboard address. Passed to JobSubmissionClient. Generated by resolve_hostname if not provided.

  • headers (dict[str, Any] | None, default: None ) –

    HTTP headers for Ray Dashboard requests. Passed to JobSubmissionClient.

  • verify (str | bool | None, default: None ) –

    Whether to verify TLS certificate. Passed to JobSubmissionClient.

  • cookies (dict[str, Any] | None, default: None ) –

    HTTP cookies for Ray Dashboard requests. Passed to JobSubmissionClient.

  • metadata (dict[str, Any] | None, default: None ) –

    Ray Job metadata. Passed to JobSubmissionClient.

Source code in src/dagster_ray/kuberay/pipes.py
def run(  # type: ignore
    self,
    *,
    context: OpOrAssetExecutionContext,
    submit_job_params: SubmitJobParams | None = None,
    ray_job: dict[str, Any] | None = None,
    extras: PipesExtras | None = None,
    address: str | None = None,
    headers: dict[str, Any] | None = None,
    verify: str | bool | None = None,
    cookies: dict[str, Any] | None = None,
    metadata: dict[str, Any] | None = None,
) -> PipesClientCompletedInvocation:
    """Execute a RayJob, enriched with the Pipes protocol.

    Parameters passed to [`JobSubmissionClient`][ray.job_submission.JobSubmissionClient] take precedence over the constructor arguments.

    At least one of ``submit_job_params`` or ``ray_job`` must be provided.

    Args:
        context: Current Dagster op or asset context.
        submit_job_params: Job submission parameters (entrypoint, runtime_env, etc.). When combined with ``ray_job``, values are merged into the template.  When used alone, a minimal manifest is auto-generated using the ``dagster/image`` run tag.
        ray_job: KubeRay RayJob template dict. Can be used alone or together with ``submit_job_params``.
        extras: Additional information to pass to the Pipes session, retrievable via [`PipesContext.get_extras`][dagster_pipes.PipesContext.get_extra].
        address: Ray dashboard address. Passed to [`JobSubmissionClient`][ray.job_submission.JobSubmissionClient].
            Generated by [`resolve_hostname`][dagster_ray.kuberay.pipes.PipesKubeRayJobClient.resolve_hostname] if not provided.
        headers: HTTP headers for Ray Dashboard requests. Passed to [`JobSubmissionClient`][ray.job_submission.JobSubmissionClient].
        verify: Whether to verify TLS certificate. Passed to [`JobSubmissionClient`][ray.job_submission.JobSubmissionClient].
        cookies: HTTP cookies for Ray Dashboard requests. Passed to [`JobSubmissionClient`][ray.job_submission.JobSubmissionClient].
        metadata: Ray Job metadata. Passed to [`JobSubmissionClient`][ray.job_submission.JobSubmissionClient].
    """
    if submit_job_params is None and ray_job is None:
        raise dg.DagsterInvariantViolationError(
            "At least one of `submit_job_params` or `ray_job` must be provided."
        )

    if ray_job is not None and submit_job_params is not None:
        ray_job = _merge_submit_params_into_ray_job(ray_job, submit_job_params)
    elif ray_job is not None:
        ray_job = copy.deepcopy(ray_job)
    else:
        ray_job = _ray_job_from_submit_params(context, submit_job_params)  # type: ignore[arg-type]

    with open_pipes_session(
        context=context,
        message_reader=self._message_reader,
        context_injector=self._context_injector,
        extras=extras,
    ) as session:
        ray_job = self._enrich_ray_job(context, session, ray_job)
        start_response = self._start(context, session, ray_job)
        start_status = cast(RayJobStatus, start_response["status"])
        ray_job_id = start_status["jobId"]  # pyright: ignore[reportTypedDictNotRequiredAccess]

        name = ray_job["metadata"]["name"]
        namespace = ray_job["metadata"]["namespace"]

        cluster_name = self.client.get_ray_cluster_name(
            name=name, namespace=namespace, timeout=self.timeout, poll_interval=self.poll_interval
        )

        resolved_address = address or self.address
        if resolved_address is None and not self.port_forward:
            cluster_status = self.client.ray_cluster_client.get_status(
                name=cluster_name, namespace=namespace, timeout=self.timeout, poll_interval=self.poll_interval
            )
            service_name = cluster_status["head"]["serviceName"]  # type: ignore
            dashboard_port = int(cluster_status["endpoints"]["dashboard"])  # type: ignore
            resolved_address = self.resolve_hostname(service_name, namespace, dashboard_port)

        with self.client.ray_cluster_client.job_submission_client(
            name=cluster_name,
            namespace=namespace,
            port_forward=self.port_forward,
            address=resolved_address,
            headers=headers or self.headers,
            verify=verify if verify is not None else self.verify,
            cookies=cookies or self.cookies,
            metadata=metadata or self.metadata,
        ) as job_submission_client:
            self._job_submission_client = job_submission_client

            session.report_launched(
                {
                    "extras": {
                        PIPES_LAUNCHED_EXTRAS_RAY_JOB_ID_KEY: ray_job_id,
                        PIPES_LAUNCHED_EXTRAS_RAY_ADDRESS_KEY: job_submission_client.get_address(),
                    }
                }
            )

            try:
                self._wait_for_completion(context, start_response)

                if isinstance(self._message_reader, PipesRayJobMessageReader) and self.port_forward:
                    # in this case the message reader will fail once port forwarding is finished
                    # TODO: merge https://github.com/danielgafni/dagster-ray/pull/123
                    # to avoid this work-around
                    self._message_reader.thread_ready.wait()
                    context.log.debug(
                        "[pipes] waiting for PipesRayJobMessageReader to complete before stopping port-forwarding"
                    )
                    self._message_reader.session_closed.set()
                    self._message_reader.completed.wait()

                return PipesClientCompletedInvocation(
                    session, metadata={"RayJob": f"{namespace}/{name}", "Ray Job ID": ray_job_id}
                )

            except DagsterExecutionInterruptedError:
                if self.forward_termination:
                    context.log.warning(
                        f"[pipes] Dagster process interrupted! Will terminate RayJob {namespace}/{name}."
                    )
                    self._terminate(
                        context,
                        start_response,
                        address=resolved_address,
                        headers=headers,
                        verify=verify,
                        cookies=cookies,
                        metadata=metadata,
                    )
                raise
resolve_hostname
resolve_hostname(service_name: str, namespace: str, dashboard_port: int) -> str

Resolve the Ray dashboard address from a Kubernetes service name, namespace, and port.

By default, returns an HTTP URL using the in-cluster FQDN (e.g. http://my-svc.namespace.svc.cluster.local:8265).

Override this method to customize the address, for example to use a proxy or a different protocol. The dashboard_port can be ignored if not needed.

Parameters:

  • service_name (str) –

    The Kubernetes service name for the Ray head node.

  • namespace (str) –

    The Kubernetes namespace.

  • dashboard_port (int) –

    The Ray dashboard port reported by the KubeRay operator.

Returns:

  • str –

    The full Ray dashboard URL (including protocol and optional port).

Example
class MyPipesKubeRayJobClient(PipesKubeRayJobClient):
    def resolve_hostname(self, service_name: str, namespace: str, dashboard_port: int) -> str:
        return f"https://{service_name}.{namespace}.company.com"
Source code in src/dagster_ray/kuberay/pipes.py
def resolve_hostname(self, service_name: str, namespace: str, dashboard_port: int) -> str:
    """Resolve the Ray dashboard address from a Kubernetes service name, namespace, and port.

    By default, returns an HTTP URL using the in-cluster FQDN
    (e.g. `http://my-svc.namespace.svc.cluster.local:8265`).

    Override this method to customize the address, for example to use a proxy
    or a different protocol. The `dashboard_port` can be ignored if not needed.

    Args:
        service_name: The Kubernetes service name for the Ray head node.
        namespace: The Kubernetes namespace.
        dashboard_port: The Ray dashboard port reported by the KubeRay operator.

    Returns:
        The full Ray dashboard URL (including protocol and optional port).

    Example:
        ```python
        class MyPipesKubeRayJobClient(PipesKubeRayJobClient):
            def resolve_hostname(self, service_name: str, namespace: str, dashboard_port: int) -> str:
                return f"https://{service_name}.{namespace}.company.com"
        ```
    """
    hostname = k8s_service_fqdn(service_name, namespace)
    return f"http://{hostname}:{dashboard_port}"

Configuration and Types

dagster_ray.kuberay.configs.RayJobConfig pydantic-model

Bases: Config

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "RayJobSpec": {
      "additionalProperties": true,
      "description": "[RayJob spec](https://ray-project.github.io/kuberay/reference/api/#rayjobspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "active_deadline_seconds": {
          "default": 86400,
          "title": "Active Deadline Seconds",
          "type": "integer"
        },
        "pre_running_deadline_seconds": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Deadline for the `RayJob` to reach the `Running` state, measured from `.status.startTime`. If it doesn't, KubeRay fails the job with reason `PreRunningDeadlineExceeded`. Useful for reaping jobs stuck in `Initializing` or `Waiting` \u2014 for example when the `RayCluster` can never be scheduled. Unset means no deadline. Requires KubeRay 1.6.0: older operators prune the field without an error.",
          "title": "Pre Running Deadline Seconds"
        },
        "backoff_limit": {
          "default": 0,
          "title": "Backoff Limit",
          "type": "integer"
        },
        "ray_cluster_spec": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterSpec"
            },
            {
              "type": "null"
            }
          ]
        },
        "submitter_pod_template": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Submitter Pod Template"
        },
        "submitter_config": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Submitter Config"
        },
        "metadata": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Metadata"
        },
        "cluster_selector": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Cluster Selector"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "deletion_strategy": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "`RayJob` cleanup policy. Unset by default: cleanup is governed by `shutdown_after_job_finishes`, which deletes the `RayCluster` once the job succeeds or fails. Requires KubeRay 1.5.0 (the field was named `deletionPolicy` before that) **and** the `RayJobDeletionPolicy` feature gate \u2014 the KubeRay controller fails the `RayJob` with `ValidationFailed` if the gate is off. The gate is alpha in 1.5.x (off by default) and beta in 1.6.x (on by default). See [Deletion Strategy](../tutorial/kuberay.md#deletion-strategy).",
          "title": "Deletion Strategy"
        },
        "runtime_env_yaml": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Runtime Env Yaml"
        },
        "job_id": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Job Id"
        },
        "submission_mode": {
          "default": "K8sJobMode",
          "enum": [
            "K8sJobMode",
            "HTTPMode",
            "InteractiveMode",
            "SidecarMode"
          ],
          "title": "Submission Mode",
          "type": "string"
        },
        "entrypoint_resources": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Resources"
        },
        "entrypoint_num_cpus": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Num Cpus"
        },
        "entrypoint_memory": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Memory"
        },
        "entrypoint_num_gpus": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Num Gpus"
        },
        "ttl_seconds_after_finished": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": 300,
          "title": "Ttl Seconds After Finished"
        },
        "shutdown_after_job_finishes": {
          "default": true,
          "title": "Shutdown After Job Finishes",
          "type": "boolean"
        },
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        }
      },
      "title": "RayJobSpec",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "properties": {
    "kind": {
      "default": "RayJob",
      "title": "Kind",
      "type": "string"
    },
    "api_version": {
      "default": "ray.io/v1",
      "title": "Api Version",
      "type": "string"
    },
    "metadata": {
      "additionalProperties": true,
      "description": "Kubernetes metadata, except the name field can be omitted. In this case it will be generated by `dagster-ray`.",
      "title": "Metadata",
      "type": "object"
    },
    "spec": {
      "$ref": "#/$defs/RayJobSpec"
    }
  },
  "title": "RayJobConfig",
  "type": "object"
}

Fields:

Attributes

metadata pydantic-field
metadata: dict[str, Any]

Kubernetes metadata, except the name field can be omitted. In this case it will be generated by dagster-ray.

spec pydantic-field
spec: RayJobSpec

Functions

to_k8s
to_k8s(
    context: AnyDagsterContext,
    image: str | None = None,
    labels: Mapping[str, str] | None = None,
    annotations: Mapping[str, str] | None = None,
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format and inject additional information

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(
    self,
    context: AnyDagsterContext,
    image: str | None = None,  # is injected into headgroup and workergroups, unless already specified there
    labels: Mapping[str, str] | None = None,
    annotations: Mapping[str, str] | None = None,
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format and inject additional information"""

    labels = labels or {}
    annotations = annotations or {}

    return {
        "apiVersion": self.api_version,
        "kind": self.kind,
        "metadata": remove_none_from_dict(
            {
                "name": self.metadata.get("name"),
                "labels": {**(self.metadata.get("labels", {}) or {}), **labels},
                "annotations": {**self.metadata.get("annotations", {}), **annotations},
            }
        ),
        "spec": self.spec.to_k8s(
            context=context,
            image=image,
            env_vars=env_vars,
        ),
    }

dagster_ray.kuberay.configs.RayJobSpec pydantic-model

Bases: PermissiveConfig

RayJob spec configuration options. A few sensible defaults are provided for convenience.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "additionalProperties": true,
  "description": "[RayJob spec](https://ray-project.github.io/kuberay/reference/api/#rayjobspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "active_deadline_seconds": {
      "default": 86400,
      "title": "Active Deadline Seconds",
      "type": "integer"
    },
    "pre_running_deadline_seconds": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Deadline for the `RayJob` to reach the `Running` state, measured from `.status.startTime`. If it doesn't, KubeRay fails the job with reason `PreRunningDeadlineExceeded`. Useful for reaping jobs stuck in `Initializing` or `Waiting` \u2014 for example when the `RayCluster` can never be scheduled. Unset means no deadline. Requires KubeRay 1.6.0: older operators prune the field without an error.",
      "title": "Pre Running Deadline Seconds"
    },
    "backoff_limit": {
      "default": 0,
      "title": "Backoff Limit",
      "type": "integer"
    },
    "ray_cluster_spec": {
      "anyOf": [
        {
          "$ref": "#/$defs/RayClusterSpec"
        },
        {
          "type": "null"
        }
      ]
    },
    "submitter_pod_template": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Submitter Pod Template"
    },
    "submitter_config": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Submitter Config"
    },
    "metadata": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Metadata"
    },
    "cluster_selector": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Cluster Selector"
    },
    "managed_by": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Managed By"
    },
    "deletion_strategy": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "`RayJob` cleanup policy. Unset by default: cleanup is governed by `shutdown_after_job_finishes`, which deletes the `RayCluster` once the job succeeds or fails. Requires KubeRay 1.5.0 (the field was named `deletionPolicy` before that) **and** the `RayJobDeletionPolicy` feature gate \u2014 the KubeRay controller fails the `RayJob` with `ValidationFailed` if the gate is off. The gate is alpha in 1.5.x (off by default) and beta in 1.6.x (on by default). See [Deletion Strategy](../tutorial/kuberay.md#deletion-strategy).",
      "title": "Deletion Strategy"
    },
    "runtime_env_yaml": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Runtime Env Yaml"
    },
    "job_id": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Job Id"
    },
    "submission_mode": {
      "default": "K8sJobMode",
      "enum": [
        "K8sJobMode",
        "HTTPMode",
        "InteractiveMode",
        "SidecarMode"
      ],
      "title": "Submission Mode",
      "type": "string"
    },
    "entrypoint_resources": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Resources"
    },
    "entrypoint_num_cpus": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Num Cpus"
    },
    "entrypoint_memory": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Memory"
    },
    "entrypoint_num_gpus": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Num Gpus"
    },
    "ttl_seconds_after_finished": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "default": 300,
      "title": "Ttl Seconds After Finished"
    },
    "shutdown_after_job_finishes": {
      "default": true,
      "title": "Shutdown After Job Finishes",
      "type": "boolean"
    },
    "suspend": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Suspend"
    }
  },
  "title": "RayJobSpec",
  "type": "object"
}

Fields:

Validators:

  • _reject_k8s_token_auth

Attributes

ray_cluster_spec pydantic-field
ray_cluster_spec: RayClusterSpec | None
entrypoint_num_cpus pydantic-field
entrypoint_num_cpus: float | None = None
entrypoint_num_gpus pydantic-field
entrypoint_num_gpus: float | None = None
entrypoint_memory pydantic-field
entrypoint_memory: float | None = None
entrypoint_resources pydantic-field
entrypoint_resources: str | None = None

Functions

to_k8s
to_k8s(
    context: AnyDagsterContext, image: str | None = None, env_vars: Mapping[str, str] | None = None
) -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format and inject additional information

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(
    self,
    context: AnyDagsterContext,
    image: str | None = None,  # is injected into headgroup and workergroups, unless already specified there
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format and inject additional information"""
    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "activeDeadlineSeconds": self.active_deadline_seconds,
                "preRunningDeadlineSeconds": self.pre_running_deadline_seconds,
                "backoffLimit": self.backoff_limit,
                "submitterPodTemplate": self.submitter_pod_template,
                "submitterConfig": self.submitter_config,
                "metadata": self.metadata,
                "clusterSelector": self.cluster_selector,
                "managedBy": self.managed_by,
                "deletionStrategy": self.deletion_strategy,
                "runtimeEnvYAML": self.runtime_env_yaml,
                "jobId": self.job_id,
                "submissionMode": self.submission_mode,
                "entrypointResources": self.entrypoint_resources,
                "entrypointNumCpus": self.entrypoint_num_cpus,
                "entrypointMemory": self.entrypoint_memory,
                "entrypointNumGpus": self.entrypoint_num_gpus,
                "ttlSecondsAfterFinished": self.ttl_seconds_after_finished,
                "shutdownAfterJobFinishes": self.shutdown_after_job_finishes,
                "suspend": self.suspend,
                "rayClusterSpec": self.ray_cluster_spec.to_k8s(context=context, image=image, env_vars=env_vars)
                if self.ray_cluster_spec is not None
                else None,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.resources.rayjob.InteractiveRayJobConfig pydantic-model

Bases: RayJobConfig

Same as RayJobConfig, but spec.submission_mode mode has to be InteractiveMode

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "InteractiveRayJobSpec": {
      "additionalProperties": true,
      "description": "Same as [`RayJobSpec`][dagster_ray.kuberay.configs.RayJobSpec], but `mode` has to be `InteractiveMode`",
      "properties": {
        "active_deadline_seconds": {
          "default": 86400,
          "title": "Active Deadline Seconds",
          "type": "integer"
        },
        "pre_running_deadline_seconds": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Deadline for the `RayJob` to reach the `Running` state, measured from `.status.startTime`. If it doesn't, KubeRay fails the job with reason `PreRunningDeadlineExceeded`. Useful for reaping jobs stuck in `Initializing` or `Waiting` \u2014 for example when the `RayCluster` can never be scheduled. Unset means no deadline. Requires KubeRay 1.6.0: older operators prune the field without an error.",
          "title": "Pre Running Deadline Seconds"
        },
        "backoff_limit": {
          "default": 0,
          "title": "Backoff Limit",
          "type": "integer"
        },
        "ray_cluster_spec": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterSpec"
            },
            {
              "type": "null"
            }
          ]
        },
        "submitter_pod_template": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Submitter Pod Template"
        },
        "submitter_config": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Submitter Config"
        },
        "metadata": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Metadata"
        },
        "cluster_selector": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Cluster Selector"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "deletion_strategy": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "`RayJob` cleanup policy. Unset by default: cleanup is governed by `shutdown_after_job_finishes`, which deletes the `RayCluster` once the job succeeds or fails. Requires KubeRay 1.5.0 (the field was named `deletionPolicy` before that) **and** the `RayJobDeletionPolicy` feature gate \u2014 the KubeRay controller fails the `RayJob` with `ValidationFailed` if the gate is off. The gate is alpha in 1.5.x (off by default) and beta in 1.6.x (on by default). See [Deletion Strategy](../tutorial/kuberay.md#deletion-strategy).",
          "title": "Deletion Strategy"
        },
        "runtime_env_yaml": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Runtime Env Yaml"
        },
        "job_id": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Job Id"
        },
        "submission_mode": {
          "const": "InteractiveMode",
          "default": "InteractiveMode",
          "title": "Submission Mode",
          "type": "string"
        },
        "entrypoint_resources": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Resources"
        },
        "entrypoint_num_cpus": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Num Cpus"
        },
        "entrypoint_memory": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Memory"
        },
        "entrypoint_num_gpus": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Entrypoint Num Gpus"
        },
        "ttl_seconds_after_finished": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": 300,
          "title": "Ttl Seconds After Finished"
        },
        "shutdown_after_job_finishes": {
          "default": true,
          "title": "Shutdown After Job Finishes",
          "type": "boolean"
        },
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        }
      },
      "title": "InteractiveRayJobSpec",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "description": "Same as [`RayJobConfig`][dagster_ray.kuberay.configs.RayJobConfig], but `spec.submission_mode` mode has to be `InteractiveMode`",
  "properties": {
    "kind": {
      "default": "RayJob",
      "title": "Kind",
      "type": "string"
    },
    "api_version": {
      "default": "ray.io/v1",
      "title": "Api Version",
      "type": "string"
    },
    "metadata": {
      "additionalProperties": true,
      "description": "Kubernetes metadata, except the name field can be omitted. In this case it will be generated by `dagster-ray`.",
      "title": "Metadata",
      "type": "object"
    },
    "spec": {
      "$ref": "#/$defs/InteractiveRayJobSpec"
    }
  },
  "title": "InteractiveRayJobConfig",
  "type": "object"
}

Fields:

Attributes

metadata pydantic-field
metadata: dict[str, Any]

Kubernetes metadata, except the name field can be omitted. In this case it will be generated by dagster-ray.

spec pydantic-field

Functions

to_k8s
to_k8s(
    context: AnyDagsterContext,
    image: str | None = None,
    labels: Mapping[str, str] | None = None,
    annotations: Mapping[str, str] | None = None,
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format and inject additional information

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(
    self,
    context: AnyDagsterContext,
    image: str | None = None,  # is injected into headgroup and workergroups, unless already specified there
    labels: Mapping[str, str] | None = None,
    annotations: Mapping[str, str] | None = None,
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format and inject additional information"""

    labels = labels or {}
    annotations = annotations or {}

    return {
        "apiVersion": self.api_version,
        "kind": self.kind,
        "metadata": remove_none_from_dict(
            {
                "name": self.metadata.get("name"),
                "labels": {**(self.metadata.get("labels", {}) or {}), **labels},
                "annotations": {**self.metadata.get("annotations", {}), **annotations},
            }
        ),
        "spec": self.spec.to_k8s(
            context=context,
            image=image,
            env_vars=env_vars,
        ),
    }

dagster_ray.kuberay.resources.rayjob.InteractiveRayJobSpec pydantic-model

Bases: RayJobSpec

Same as RayJobSpec, but mode has to be InteractiveMode

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "additionalProperties": true,
  "description": "Same as [`RayJobSpec`][dagster_ray.kuberay.configs.RayJobSpec], but `mode` has to be `InteractiveMode`",
  "properties": {
    "active_deadline_seconds": {
      "default": 86400,
      "title": "Active Deadline Seconds",
      "type": "integer"
    },
    "pre_running_deadline_seconds": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Deadline for the `RayJob` to reach the `Running` state, measured from `.status.startTime`. If it doesn't, KubeRay fails the job with reason `PreRunningDeadlineExceeded`. Useful for reaping jobs stuck in `Initializing` or `Waiting` \u2014 for example when the `RayCluster` can never be scheduled. Unset means no deadline. Requires KubeRay 1.6.0: older operators prune the field without an error.",
      "title": "Pre Running Deadline Seconds"
    },
    "backoff_limit": {
      "default": 0,
      "title": "Backoff Limit",
      "type": "integer"
    },
    "ray_cluster_spec": {
      "anyOf": [
        {
          "$ref": "#/$defs/RayClusterSpec"
        },
        {
          "type": "null"
        }
      ]
    },
    "submitter_pod_template": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Submitter Pod Template"
    },
    "submitter_config": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Submitter Config"
    },
    "metadata": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Metadata"
    },
    "cluster_selector": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Cluster Selector"
    },
    "managed_by": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Managed By"
    },
    "deletion_strategy": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "`RayJob` cleanup policy. Unset by default: cleanup is governed by `shutdown_after_job_finishes`, which deletes the `RayCluster` once the job succeeds or fails. Requires KubeRay 1.5.0 (the field was named `deletionPolicy` before that) **and** the `RayJobDeletionPolicy` feature gate \u2014 the KubeRay controller fails the `RayJob` with `ValidationFailed` if the gate is off. The gate is alpha in 1.5.x (off by default) and beta in 1.6.x (on by default). See [Deletion Strategy](../tutorial/kuberay.md#deletion-strategy).",
      "title": "Deletion Strategy"
    },
    "runtime_env_yaml": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Runtime Env Yaml"
    },
    "job_id": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Job Id"
    },
    "submission_mode": {
      "const": "InteractiveMode",
      "default": "InteractiveMode",
      "title": "Submission Mode",
      "type": "string"
    },
    "entrypoint_resources": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Resources"
    },
    "entrypoint_num_cpus": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Num Cpus"
    },
    "entrypoint_memory": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Memory"
    },
    "entrypoint_num_gpus": {
      "anyOf": [
        {
          "type": "number"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Entrypoint Num Gpus"
    },
    "ttl_seconds_after_finished": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "default": 300,
      "title": "Ttl Seconds After Finished"
    },
    "shutdown_after_job_finishes": {
      "default": true,
      "title": "Shutdown After Job Finishes",
      "type": "boolean"
    },
    "suspend": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Suspend"
    }
  },
  "title": "InteractiveRayJobSpec",
  "type": "object"
}

Fields:

Validators:

  • _reject_k8s_token_auth

Attributes

submission_mode pydantic-field
submission_mode: Literal['InteractiveMode'] = 'InteractiveMode'
active_deadline_seconds pydantic-field
active_deadline_seconds: int = 60 * 60 * 24
pre_running_deadline_seconds pydantic-field
pre_running_deadline_seconds: int | None = None

Deadline for the RayJob to reach the Running state, measured from .status.startTime. If it doesn't, KubeRay fails the job with reason PreRunningDeadlineExceeded. Useful for reaping jobs stuck in Initializing or Waiting — for example when the RayCluster can never be scheduled. Unset means no deadline. Requires KubeRay 1.6.0: older operators prune the field without an error.

backoff_limit pydantic-field
backoff_limit: int = 0
ray_cluster_spec pydantic-field
ray_cluster_spec: RayClusterSpec | None
submitter_pod_template pydantic-field
submitter_pod_template: dict[str, Any] | None = None
submitter_config pydantic-field
submitter_config: dict[str, Any] | None = None
metadata pydantic-field
metadata: dict[str, Any] | None = None
cluster_selector pydantic-field
cluster_selector: dict[str, str] | None = None
managed_by pydantic-field
managed_by: str | None = None
deletion_strategy pydantic-field
deletion_strategy: dict[str, Any] | None = None

RayJob cleanup policy. Unset by default: cleanup is governed by shutdown_after_job_finishes, which deletes the RayCluster once the job succeeds or fails. Requires KubeRay 1.5.0 (the field was named deletionPolicy before that) and the RayJobDeletionPolicy feature gate — the KubeRay controller fails the RayJob with ValidationFailed if the gate is off. The gate is alpha in 1.5.x (off by default) and beta in 1.6.x (on by default). See Deletion Strategy.

runtime_env_yaml pydantic-field
runtime_env_yaml: str | None = None
job_id pydantic-field
job_id: str | None = None
entrypoint_resources pydantic-field
entrypoint_resources: str | None = None
entrypoint_num_cpus pydantic-field
entrypoint_num_cpus: float | None = None
entrypoint_memory pydantic-field
entrypoint_memory: float | None = None
entrypoint_num_gpus pydantic-field
entrypoint_num_gpus: float | None = None
ttl_seconds_after_finished pydantic-field
ttl_seconds_after_finished: int | None = 5 * 60
shutdown_after_job_finishes pydantic-field
shutdown_after_job_finishes: bool = True
suspend pydantic-field
suspend: bool | None = None

Functions

to_k8s
to_k8s(
    context: AnyDagsterContext, image: str | None = None, env_vars: Mapping[str, str] | None = None
) -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format and inject additional information

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(
    self,
    context: AnyDagsterContext,
    image: str | None = None,  # is injected into headgroup and workergroups, unless already specified there
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format and inject additional information"""
    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "activeDeadlineSeconds": self.active_deadline_seconds,
                "preRunningDeadlineSeconds": self.pre_running_deadline_seconds,
                "backoffLimit": self.backoff_limit,
                "submitterPodTemplate": self.submitter_pod_template,
                "submitterConfig": self.submitter_config,
                "metadata": self.metadata,
                "clusterSelector": self.cluster_selector,
                "managedBy": self.managed_by,
                "deletionStrategy": self.deletion_strategy,
                "runtimeEnvYAML": self.runtime_env_yaml,
                "jobId": self.job_id,
                "submissionMode": self.submission_mode,
                "entrypointResources": self.entrypoint_resources,
                "entrypointNumCpus": self.entrypoint_num_cpus,
                "entrypointMemory": self.entrypoint_memory,
                "entrypointNumGpus": self.entrypoint_num_gpus,
                "ttlSecondsAfterFinished": self.ttl_seconds_after_finished,
                "shutdownAfterJobFinishes": self.shutdown_after_job_finishes,
                "suspend": self.suspend,
                "rayClusterSpec": self.ray_cluster_spec.to_k8s(context=context, image=image, env_vars=env_vars)
                if self.ray_cluster_spec is not None
                else None,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.configs.RayClusterConfig pydantic-model

Bases: Config

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterSpec": {
      "additionalProperties": true,
      "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "suspend": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Suspend"
        },
        "managed_by": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Managed By"
        },
        "autoscaler_options": {
          "additionalProperties": true,
          "default": {
            "upscalingMode": "Default",
            "idleTimeoutSeconds": 60,
            "env": [],
            "envFrom": [],
            "resources": {
              "limits": {
                "cpu": "50m",
                "memory": "0.1Gi"
              },
              "requests": {
                "cpu": "50m",
                "memory": "0.1Gi"
              }
            }
          },
          "title": "Autoscaler Options",
          "type": "object"
        },
        "head_service_annotations": {
          "anyOf": [
            {
              "additionalProperties": {
                "type": "string"
              },
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Head Service Annotations"
        },
        "enable_in_tree_autoscaling": {
          "default": false,
          "title": "Enable In Tree Autoscaling",
          "type": "boolean"
        },
        "gcs_fault_tolerance_options": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gcs Fault Tolerance Options"
        },
        "head_group_spec": {
          "additionalProperties": true,
          "default": {
            "serviceType": "ClusterIP",
            "rayStartParams": {},
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "template": {
              "spec": {
                "affinity": {},
                "containers": [
                  {
                    "imagePullPolicy": "Always",
                    "name": "head",
                    "volumeMounts": [
                      {
                        "mountPath": "/tmp/ray",
                        "name": "ray-logs"
                      }
                    ]
                  }
                ],
                "imagePullSecrets": [],
                "nodeSelector": {},
                "tolerations": [],
                "volumes": [
                  {
                    "emptyDir": {},
                    "name": "ray-logs"
                  }
                ]
              }
            }
          },
          "title": "Head Group Spec",
          "type": "object"
        },
        "ray_version": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Ray Version"
        },
        "worker_group_specs": {
          "default": [
            {
              "groupName": "workers",
              "replicas": 0,
              "minReplicas": 0,
              "maxReplicas": 1,
              "rayStartParams": {},
              "template": {
                "metadata": {
                  "annotations": {},
                  "labels": {}
                },
                "spec": {
                  "affinity": {},
                  "containers": [
                    {
                      "imagePullPolicy": "Always",
                      "name": "worker",
                      "volumeMounts": [
                        {
                          "mountPath": "/tmp/ray",
                          "name": "ray-logs"
                        }
                      ]
                    }
                  ],
                  "imagePullSecrets": [],
                  "nodeSelector": {},
                  "tolerations": [],
                  "volumes": [
                    {
                      "emptyDir": {},
                      "name": "ray-logs"
                    }
                  ]
                }
              }
            }
          ],
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "title": "Worker Group Specs",
          "type": "array"
        },
        "auth_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/AuthOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null
        },
        "upgrade_strategy": {
          "anyOf": [
            {
              "$ref": "#/$defs/RayClusterUpgradeStrategy"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
        },
        "tls_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/TLSOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
        },
        "network_policy": {
          "anyOf": [
            {
              "$ref": "#/$defs/NetworkPolicyConfig"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
        },
        "history_server_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/HistoryServerOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
        }
      },
      "title": "RayClusterSpec",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "properties": {
    "kind": {
      "default": "RayCluster",
      "title": "Kind",
      "type": "string"
    },
    "api_version": {
      "default": "ray.io/v1",
      "title": "Api Version",
      "type": "string"
    },
    "metadata": {
      "additionalProperties": true,
      "description": "Kubernetes metadata, except the name field can be omitted. In this case it will be generated by `dagster-ray`.",
      "title": "Metadata",
      "type": "object"
    },
    "spec": {
      "$ref": "#/$defs/RayClusterSpec"
    }
  },
  "title": "RayClusterConfig",
  "type": "object"
}

Fields:

Attributes

metadata pydantic-field
metadata: dict[str, Any]

Kubernetes metadata, except the name field can be omitted. In this case it will be generated by dagster-ray.

spec pydantic-field

Functions

to_k8s
to_k8s(
    context: AnyDagsterContext,
    image: str | None = None,
    labels: Mapping[str, str] | None = None,
    annotations: Mapping[str, str] | None = None,
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]
Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(
    self,
    context: AnyDagsterContext,
    image: str | None = None,  # is injected into headgroup and workergroups, unless already specified there
    labels: Mapping[str, str] | None = None,
    annotations: Mapping[str, str] | None = None,
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]:
    assert context.log is not None
    """Convert into Kubernetes manifests in camelCase format and inject additional information"""

    labels = labels or {}
    annotations = annotations or {}

    return {
        "apiVersion": self.api_version,
        "kind": self.kind,
        "metadata": remove_none_from_dict(
            {
                "name": self.metadata.get("name"),
                "labels": {**(self.metadata.get("labels", {}) or {}), **labels},
                "annotations": {**self.metadata.get("annotations", {}), **annotations},
            }
        ),
        "spec": self.spec.to_k8s(context=context, image=image, env_vars=env_vars),
    }

dagster_ray.kuberay.configs.RayClusterSpec pydantic-model

Bases: PermissiveConfig

RayCluster spec configuration options. A few sensible defaults are provided for convenience.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "$defs": {
    "AuthOptions": {
      "additionalProperties": true,
      "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "default": "token",
          "enum": [
            "token",
            "disabled"
          ],
          "title": "Mode",
          "type": "string"
        },
        "secret_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
          "title": "Secret Name"
        },
        "enable_k8s_token_auth": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
          "title": "Enable K8S Token Auth"
        }
      },
      "title": "AuthOptions",
      "type": "object"
    },
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    },
    "HistoryServerOptions": {
      "additionalProperties": true,
      "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "collector_options": {
          "anyOf": [
            {
              "$ref": "#/$defs/CollectorOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
        }
      },
      "title": "HistoryServerOptions",
      "type": "object"
    },
    "NetworkPolicyConfig": {
      "additionalProperties": true,
      "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "mode": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
          "title": "Mode"
        },
        "head": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
          "title": "Head"
        },
        "worker": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
          "title": "Worker"
        },
        "worker_groups": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
          "title": "Worker Groups"
        }
      },
      "title": "NetworkPolicyConfig",
      "type": "object"
    },
    "RayClusterUpgradeStrategy": {
      "additionalProperties": true,
      "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "type": {
          "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
          "enum": [
            "Recreate",
            "None"
          ],
          "title": "Type",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "title": "RayClusterUpgradeStrategy",
      "type": "object"
    },
    "TLSOptions": {
      "additionalProperties": true,
      "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "enabled": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
          "title": "Enabled"
        }
      },
      "title": "TLSOptions",
      "type": "object"
    }
  },
  "additionalProperties": true,
  "description": "[RayCluster spec](https://ray-project.github.io/kuberay/reference/api/#rayclusterspec) configuration options. A few sensible defaults are provided for convenience.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "suspend": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Suspend"
    },
    "managed_by": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Managed By"
    },
    "autoscaler_options": {
      "additionalProperties": true,
      "default": {
        "upscalingMode": "Default",
        "idleTimeoutSeconds": 60,
        "env": [],
        "envFrom": [],
        "resources": {
          "limits": {
            "cpu": "50m",
            "memory": "0.1Gi"
          },
          "requests": {
            "cpu": "50m",
            "memory": "0.1Gi"
          }
        }
      },
      "title": "Autoscaler Options",
      "type": "object"
    },
    "head_service_annotations": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Head Service Annotations"
    },
    "enable_in_tree_autoscaling": {
      "default": false,
      "title": "Enable In Tree Autoscaling",
      "type": "boolean"
    },
    "gcs_fault_tolerance_options": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Gcs Fault Tolerance Options"
    },
    "head_group_spec": {
      "additionalProperties": true,
      "default": {
        "serviceType": "ClusterIP",
        "rayStartParams": {},
        "metadata": {
          "annotations": {},
          "labels": {}
        },
        "template": {
          "spec": {
            "affinity": {},
            "containers": [
              {
                "imagePullPolicy": "Always",
                "name": "head",
                "volumeMounts": [
                  {
                    "mountPath": "/tmp/ray",
                    "name": "ray-logs"
                  }
                ]
              }
            ],
            "imagePullSecrets": [],
            "nodeSelector": {},
            "tolerations": [],
            "volumes": [
              {
                "emptyDir": {},
                "name": "ray-logs"
              }
            ]
          }
        }
      },
      "title": "Head Group Spec",
      "type": "object"
    },
    "ray_version": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Ray Version"
    },
    "worker_group_specs": {
      "default": [
        {
          "groupName": "workers",
          "replicas": 0,
          "minReplicas": 0,
          "maxReplicas": 1,
          "rayStartParams": {},
          "template": {
            "metadata": {
              "annotations": {},
              "labels": {}
            },
            "spec": {
              "affinity": {},
              "containers": [
                {
                  "imagePullPolicy": "Always",
                  "name": "worker",
                  "volumeMounts": [
                    {
                      "mountPath": "/tmp/ray",
                      "name": "ray-logs"
                    }
                  ]
                }
              ],
              "imagePullSecrets": [],
              "nodeSelector": {},
              "tolerations": [],
              "volumes": [
                {
                  "emptyDir": {},
                  "name": "ray-logs"
                }
              ]
            }
          }
        }
      ],
      "items": {
        "additionalProperties": true,
        "type": "object"
      },
      "title": "Worker Group Specs",
      "type": "array"
    },
    "auth_options": {
      "anyOf": [
        {
          "$ref": "#/$defs/AuthOptions"
        },
        {
          "type": "null"
        }
      ],
      "default": null
    },
    "upgrade_strategy": {
      "anyOf": [
        {
          "$ref": "#/$defs/RayClusterUpgradeStrategy"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Scaling policy used when upgrading the `RayCluster`. See [RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy). Requires KubeRay 1.6.0: older operators prune the field without an error."
    },
    "tls_options": {
      "anyOf": [
        {
          "$ref": "#/$defs/TLSOptions"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Mutual TLS settings for the `RayCluster`. See [TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions)."
    },
    "network_policy": {
      "anyOf": [
        {
          "$ref": "#/$defs/NetworkPolicyConfig"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Network isolation settings for the `RayCluster`. See [NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig)."
    },
    "history_server_options": {
      "anyOf": [
        {
          "$ref": "#/$defs/HistoryServerOptions"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "History server collector sidecar settings for the `RayCluster`. See [HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions)."
    }
  },
  "title": "RayClusterSpec",
  "type": "object"
}

Fields:

Validators:

  • _validate_auth_options

Attributes

suspend pydantic-field
suspend: bool | None = None
managed_by pydantic-field
managed_by: str | None = None
autoscaler_options pydantic-field
autoscaler_options: dict[str, Any] = DEFAULT_AUTOSCALER_OPTIONS
head_service_annotations pydantic-field
head_service_annotations: dict[str, str] | None = None
enable_in_tree_autoscaling pydantic-field
enable_in_tree_autoscaling: bool = False
gcs_fault_tolerance_options pydantic-field
gcs_fault_tolerance_options: dict[str, Any] | None = None
head_group_spec pydantic-field
head_group_spec: dict[str, Any] = DEFAULT_HEAD_GROUP_SPEC
ray_version pydantic-field
ray_version: str | None = None
worker_group_specs pydantic-field
worker_group_specs: list[dict[str, Any]] = DEFAULT_WORKER_GROUP_SPECS
auth_options pydantic-field
auth_options: AuthOptions | None = None
upgrade_strategy pydantic-field
upgrade_strategy: RayClusterUpgradeStrategy | None = None

Scaling policy used when upgrading the RayCluster. See RayClusterUpgradeStrategy. Requires KubeRay 1.6.0: older operators prune the field without an error.

tls_options pydantic-field
tls_options: TLSOptions | None = None

Mutual TLS settings for the RayCluster. See TLSOptions.

network_policy pydantic-field
network_policy: NetworkPolicyConfig | None = None

Network isolation settings for the RayCluster. See NetworkPolicyConfig.

history_server_options pydantic-field
history_server_options: HistoryServerOptions | None = None

History server collector sidecar settings for the RayCluster. See HistoryServerOptions.

Functions

to_k8s
to_k8s(
    context: AnyDagsterContext, image: str | None = None, env_vars: Mapping[str, str] | None = None
) -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format and inject additional information

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(
    self,
    context: AnyDagsterContext,
    image: str | None = None,  # is injected into headgroup and workergroups, unless already specified there
    env_vars: Mapping[str, str] | None = None,
) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format and inject additional information"""

    assert context.log is not None

    # TODO: inject self.redis_port and self.dashboard_port into the RayCluster configuration
    # TODO: auto-apply some tags from dagster-k8s/config

    head_group_spec = self.head_group_spec.copy()
    worker_group_specs = self.worker_group_specs.copy()

    k8s_env_vars: list[dict[str, Any]] = []

    if env_vars:
        for key, value in env_vars.items():
            k8s_env_vars.append({"name": key, "value": value})

    def update_group_spec(group_spec: dict[str, Any]):
        # TODO: only inject if the container has a `dagster.io/inject-image` annotation or smth
        if group_spec["template"]["spec"]["containers"][0].get("image") is None:
            if image is None:
                raise ValueError(MISSING_IMAGE_MESSAGE)
            else:
                group_spec["template"]["spec"]["containers"][0]["image"] = image

        for container in group_spec["template"]["spec"]["containers"]:
            container["env"] = container.get("env", []) + k8s_env_vars

    update_group_spec(head_group_spec)
    for worker_group_spec in worker_group_specs:
        update_group_spec(worker_group_spec)

    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "enableInTreeAutoscaling": self.enable_in_tree_autoscaling,
                "autoscalerOptions": self.autoscaler_options,
                "headGroupSpec": head_group_spec,
                "workerGroupSpecs": worker_group_specs,
                "suspend": self.suspend,
                "managedBy": self.managed_by,
                "headServiceAnnotations": self.head_service_annotations,
                "gcsFaultToleranceOptions": self.gcs_fault_tolerance_options,
                "rayVersion": self.ray_version,
                "authOptions": self.auth_options.to_k8s() if self.auth_options is not None else None,
                "upgradeStrategy": self.upgrade_strategy.to_k8s() if self.upgrade_strategy is not None else None,
                "tlsOptions": self.tls_options.to_k8s() if self.tls_options is not None else None,
                "networkPolicy": self.network_policy.to_k8s() if self.network_policy is not None else None,
                "historyServerOptions": self.history_server_options.to_k8s()
                if self.history_server_options is not None
                else None,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.configs.AuthOptions pydantic-model

Bases: PermissiveConfig

AuthOptions for the Ray cluster.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "additionalProperties": true,
  "description": "[AuthOptions](https://ray-project.github.io/kuberay/reference/api/#authoptions) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "mode": {
      "default": "token",
      "enum": [
        "token",
        "disabled"
      ],
      "title": "Mode",
      "type": "string"
    },
    "secret_name": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Name of the `Secret` holding the authentication token. The `Secret` must have an `auth_token` data key. If set, KubeRay skips generating a per-`RayCluster` token `Secret`. Requires KubeRay 1.6.0.",
      "title": "Secret Name"
    },
    "enable_k8s_token_auth": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Delegate authentication to the Kubernetes API server. Sets `RAY_ENABLE_K8S_TOKEN_AUTH=true` on all Ray pods; the ServiceAccount token mounted into Raylets must be granted the `ray:write` custom verb via RBAC. Requires `mode='token'`, Ray 2.55.0 or later, and KubeRay 1.6.0, and cannot be combined with `secret_name` \u2014 the token comes from the mounted ServiceAccount rather than a `Secret`. KubeRay rejects it for `RayJob` and `RayService`, so it cannot be used with [`KubeRayInteractiveJob`][dagster_ray.kuberay.KubeRayInteractiveJob] \u2014 use [`KubeRayCluster`][dagster_ray.kuberay.KubeRayCluster].",
      "title": "Enable K8S Token Auth"
    }
  },
  "title": "AuthOptions",
  "type": "object"
}

Fields:

Attributes

mode pydantic-field
mode: Literal['token', 'disabled'] = 'token'
secret_name pydantic-field
secret_name: str | None = None

Name of the Secret holding the authentication token. The Secret must have an auth_token data key. If set, KubeRay skips generating a per-RayCluster token Secret. Requires KubeRay 1.6.0.

enable_k8s_token_auth pydantic-field
enable_k8s_token_auth: bool | None = None

Functions

to_k8s
to_k8s() -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(self) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format"""
    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "mode": self.mode,
                "secretName": self.secret_name,
                "enableK8sTokenAuth": self.enable_k8s_token_auth,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.configs.RayClusterUpgradeStrategy pydantic-model

Bases: PermissiveConfig

RayClusterUpgradeStrategy for the Ray cluster.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "additionalProperties": true,
  "description": "[RayClusterUpgradeStrategy](https://ray-project.github.io/kuberay/reference/api/#rayclusterupgradestrategy) for the Ray cluster.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "type": {
      "description": "Strategy used when upgrading the `RayCluster` pods. `Recreate` deletes all existing pods before creating new ones; the string `\"None\"` creates no new pods. Note that `\"None\"` is a KubeRay strategy name, distinct from leaving `upgrade_strategy` itself unset.",
      "enum": [
        "Recreate",
        "None"
      ],
      "title": "Type",
      "type": "string"
    }
  },
  "required": [
    "type"
  ],
  "title": "RayClusterUpgradeStrategy",
  "type": "object"
}

Fields:

Attributes

type pydantic-field
type: Literal['Recreate', 'None']

Strategy used when upgrading the RayCluster pods. Recreate deletes all existing pods before creating new ones; the string "None" creates no new pods. Note that "None" is a KubeRay strategy name, distinct from leaving upgrade_strategy itself unset.

Functions

to_k8s
to_k8s() -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(self) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format"""
    return merge_extra_k8s_fields(remove_none_from_dict({"type": self.type}), self.model_extra)

dagster_ray.kuberay.configs.TLSOptions pydantic-model

Bases: PermissiveConfig

TLSOptions for the Ray cluster.

Alpha in KubeRay 1.7.0

Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later. Older operators prune the field without an error.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "additionalProperties": true,
  "description": "[TLSOptions](https://ray-project.github.io/kuberay/reference/api/#tlsoptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's TLS feature gate, cert-manager, and Ray 2.55.1 or later.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "enabled": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Enable mutual TLS between the `RayCluster` pods, issued via cert-manager.",
      "title": "Enabled"
    }
  },
  "title": "TLSOptions",
  "type": "object"
}

Fields:

Attributes

enabled pydantic-field
enabled: bool | None = None

Enable mutual TLS between the RayCluster pods, issued via cert-manager.

Functions

to_k8s
to_k8s() -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(self) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format"""
    return merge_extra_k8s_fields(remove_none_from_dict({"enabled": self.enabled}), self.model_extra)

dagster_ray.kuberay.configs.NetworkPolicyConfig pydantic-model

Bases: PermissiveConfig

NetworkPolicyConfig for the Ray cluster.

Alpha in KubeRay 1.7.0

Requires the operator's network policy feature gate. Older operators prune the field without an error.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "additionalProperties": true,
  "description": "[NetworkPolicyConfig](https://ray-project.github.io/kuberay/reference/api/#networkpolicyconfig) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's network policy feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "mode": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Network isolation level, one of `DenyAll`, `DenyAllIngress`, or `DenyAllEgress`. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to `DenyAll` when omitted. Typed as `str` because Dagster's config system cannot resolve an optional `Literal`.",
      "title": "Mode"
    },
    "head": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Custom `NetworkPolicyRules` (`ingressRules`/`egressRules`) applied only to the head pod's policy.",
      "title": "Head"
    },
    "worker": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Custom `NetworkPolicyRules` applied to every worker pod's policy.",
      "title": "Worker"
    },
    "worker_groups": {
      "anyOf": [
        {
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "type": "array"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Per-worker-group `NetworkPolicyRules`, each keyed by its `groupName`.",
      "title": "Worker Groups"
    }
  },
  "title": "NetworkPolicyConfig",
  "type": "object"
}

Fields:

Attributes

mode pydantic-field
mode: str | None = None

Network isolation level, one of DenyAll, DenyAllIngress, or DenyAllEgress. All modes allow intra-cluster pod-to-pod traffic. KubeRay defaults it to DenyAll when omitted. Typed as str because Dagster's config system cannot resolve an optional Literal.

head pydantic-field
head: dict[str, Any] | None = None

Custom NetworkPolicyRules (ingressRules/egressRules) applied only to the head pod's policy.

worker pydantic-field
worker: dict[str, Any] | None = None

Custom NetworkPolicyRules applied to every worker pod's policy.

worker_groups pydantic-field
worker_groups: list[dict[str, Any]] | None = None

Per-worker-group NetworkPolicyRules, each keyed by its groupName.

Functions

to_k8s
to_k8s() -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(self) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format"""
    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "mode": self.mode,
                "head": self.head,
                "worker": self.worker,
                "workerGroups": self.worker_groups,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.configs.HistoryServerOptions pydantic-model

Bases: PermissiveConfig

HistoryServerOptions for the Ray cluster.

Alpha in KubeRay 1.7.0

Requires the operator's history server feature gate. Older operators prune the field without an error.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "$defs": {
    "CollectorOptions": {
      "additionalProperties": true,
      "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
      "properties": {
        "image": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image"
        },
        "image_pull_policy": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Image Pull Policy"
        },
        "resources": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Resources"
        },
        "env": {
          "anyOf": [
            {
              "items": {
                "additionalProperties": true,
                "type": "object"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Env"
        }
      },
      "title": "CollectorOptions",
      "type": "object"
    }
  },
  "additionalProperties": true,
  "description": "[HistoryServerOptions](https://ray-project.github.io/kuberay/reference/api/#historyserveroptions) for the Ray cluster.\n\n!!! warning \"Alpha in KubeRay 1.7.0\"\n    Requires the operator's history server feature gate.\n    Older operators prune the field without an error.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "collector_options": {
      "anyOf": [
        {
          "$ref": "#/$defs/CollectorOptions"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Configuration for the history server event-collector sidecar injected into the Ray pods."
    }
  },
  "title": "HistoryServerOptions",
  "type": "object"
}

Fields:

Attributes

collector_options pydantic-field
collector_options: CollectorOptions | None = None

Configuration for the history server event-collector sidecar injected into the Ray pods.

Functions

to_k8s
to_k8s() -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(self) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format"""
    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "collectorOptions": self.collector_options.to_k8s() if self.collector_options is not None else None,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.configs.CollectorOptions pydantic-model

Bases: PermissiveConfig

CollectorOptions for the history server collector sidecar.

Every field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either snake_case or camelCase. See Extra Spec Fields.

Show JSON schema:
{
  "additionalProperties": true,
  "description": "[CollectorOptions](https://ray-project.github.io/kuberay/reference/api/#collectoroptions) for the history server collector sidecar.\n\nEvery field the CRD supports is meant to be declared here. As an escape hatch for any that is missing, undeclared fields are passed through to the Kubernetes manifest, in either `snake_case` or `camelCase`. See [Extra Spec Fields](../tutorial/kuberay.md#extra-spec-fields).",
  "properties": {
    "image": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Image"
    },
    "image_pull_policy": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Image Pull Policy"
    },
    "resources": {
      "anyOf": [
        {
          "additionalProperties": true,
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Resources"
    },
    "env": {
      "anyOf": [
        {
          "items": {
            "additionalProperties": true,
            "type": "object"
          },
          "type": "array"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Env"
    }
  },
  "title": "CollectorOptions",
  "type": "object"
}

Fields:

Attributes

image pydantic-field
image: str | None = None
image_pull_policy pydantic-field
image_pull_policy: str | None = None
resources pydantic-field
resources: dict[str, Any] | None = None
env pydantic-field
env: list[dict[str, Any]] | None = None

Functions

to_k8s
to_k8s() -> dict[str, Any]

Convert into Kubernetes manifests in camelCase format

Source code in src/dagster_ray/kuberay/configs.py
def to_k8s(self) -> dict[str, Any]:
    """Convert into Kubernetes manifests in camelCase format"""
    return merge_extra_k8s_fields(
        remove_none_from_dict(
            {
                "image": self.image,
                "imagePullPolicy": self.image_pull_policy,
                "resources": self.resources,
                "env": self.env,
            }
        ),
        self.model_extra,
    )

dagster_ray.kuberay.configs.MatchDagsterLabels pydantic-model

Bases: Config

Show JSON schema:
{
  "properties": {
    "cluster_sharing": {
      "default": true,
      "description": "Whether to match on `dagster/cluster-sharing=true` label.",
      "title": "Cluster Sharing",
      "type": "boolean"
    },
    "code_location": {
      "default": true,
      "description": "Whether to match on `dagster/code-location` label. The value will be taken from the current Dagster code location.",
      "title": "Code Location",
      "type": "boolean"
    },
    "resource_key": {
      "default": true,
      "description": "Whether to match on `dagster/resource-key` label. The value will be taken from the current Dagster resource key.",
      "title": "Resource Key",
      "type": "boolean"
    },
    "git_sha": {
      "default": true,
      "description": "Whether to match on `dagster/git-sha` label. The value will be taken from `DAGSTER_CLOUD_GIT_SHA` environment variable.",
      "title": "Git Sha",
      "type": "boolean"
    },
    "run_id": {
      "default": false,
      "description": "Whether to match on `dagster/run-id` label. The value will be taken from the current Dagster run ID.",
      "title": "Run Id",
      "type": "boolean"
    }
  },
  "title": "MatchDagsterLabels",
  "type": "object"
}

Fields:

Attributes

cluster_sharing pydantic-field
cluster_sharing: bool = True

Whether to match on dagster/cluster-sharing=true label.

code_location pydantic-field
code_location: bool = True

Whether to match on dagster/code-location label. The value will be taken from the current Dagster code location.

resource_key pydantic-field
resource_key: bool = True

Whether to match on dagster/resource-key label. The value will be taken from the current Dagster resource key.

git_sha pydantic-field
git_sha: bool = True

Whether to match on dagster/git-sha label. The value will be taken from DAGSTER_CLOUD_GIT_SHA environment variable.

run_id pydantic-field
run_id: bool = False

Whether to match on dagster/run-id label. The value will be taken from the current Dagster run ID.

dagster_ray.kuberay.configs.ClusterSharing pydantic-model

Bases: Config

Defines the strategy for sharing RayCluster resources with other Dagster steps.

By default, the cluster is expected to be created by Dagster during one of the previously executed steps.

Note

Cluster sharing uses the Kubernetes Lease API (coordination.k8s.io) for leader election to coordinate cluster creation across parallel steps. The Dagster ServiceAccount must have create, get, and delete permissions on leases in the coordination.k8s.io API group.

Show JSON schema:
{
  "$defs": {
    "ClusterSharingHeartbeat": {
      "description": "Controls background renewal of the cluster sharing lock while the Dagster step is running.\n\nWithout renewal, the lock expires `ttl_seconds` after step start and the garbage collection\nsensor may delete the cluster while the step is still using it. With renewal, `ttl_seconds`\ncan stay short: idle clusters are reaped promptly, active steps survive.\n\nEach renewal updates the lock's `heartbeat_at` timestamp; `created_at` always points at the\ninitial lock placement. A hanging step renews its lock indefinitely \u2014 set the\n`dagster/max_runtime` tag on runs to bound step runtime.",
      "properties": {
        "enabled": {
          "default": true,
          "description": "Whether to renew the cluster sharing lock in the background while the Dagster step is running.",
          "title": "Enabled",
          "type": "boolean"
        },
        "refresh_seconds": {
          "default": 10.0,
          "description": "How often to renew the lock. Must be well below `ClusterSharing.ttl_seconds` \u2014 if a renewal is missed, the lock must not have expired yet, or the cluster can be deleted mid-step.",
          "title": "Refresh Seconds",
          "type": "number"
        }
      },
      "title": "ClusterSharingHeartbeat",
      "type": "object"
    },
    "MatchDagsterLabels": {
      "properties": {
        "cluster_sharing": {
          "default": true,
          "description": "Whether to match on `dagster/cluster-sharing=true` label.",
          "title": "Cluster Sharing",
          "type": "boolean"
        },
        "code_location": {
          "default": true,
          "description": "Whether to match on `dagster/code-location` label. The value will be taken from the current Dagster code location.",
          "title": "Code Location",
          "type": "boolean"
        },
        "resource_key": {
          "default": true,
          "description": "Whether to match on `dagster/resource-key` label. The value will be taken from the current Dagster resource key.",
          "title": "Resource Key",
          "type": "boolean"
        },
        "git_sha": {
          "default": true,
          "description": "Whether to match on `dagster/git-sha` label. The value will be taken from `DAGSTER_CLOUD_GIT_SHA` environment variable.",
          "title": "Git Sha",
          "type": "boolean"
        },
        "run_id": {
          "default": false,
          "description": "Whether to match on `dagster/run-id` label. The value will be taken from the current Dagster run ID.",
          "title": "Run Id",
          "type": "boolean"
        }
      },
      "title": "MatchDagsterLabels",
      "type": "object"
    }
  },
  "description": "Defines the strategy for sharing `RayCluster` resources with other Dagster steps.\n\nBy default, the cluster is expected to be created by Dagster during one of the previously executed steps.\n\n!!!note\n\n    Cluster sharing uses the Kubernetes Lease API (`coordination.k8s.io`) for leader election\n    to coordinate cluster creation across parallel steps. The Dagster ServiceAccount must have\n    `create`, `get`, and `delete` permissions on `leases` in the `coordination.k8s.io` API group.",
  "properties": {
    "enabled": {
      "default": false,
      "description": "Whether to enable sharing of RayClusters.",
      "title": "Enabled",
      "type": "boolean"
    },
    "match_dagster_labels": {
      "$ref": "#/$defs/MatchDagsterLabels",
      "description": "Configuration for matching on Dagster-generated labels."
    },
    "match_labels": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Additional user-provided labels to match on.",
      "title": "Match Labels"
    },
    "ttl_seconds": {
      "default": 1800.0,
      "description": "Time to live for the lock placed on the `RayCluster` resource, marking it as in use by the current Dagster step. The lock is renewed periodically while the step is running (see `heartbeat`), so this only needs to cover the gap between renewals.",
      "title": "Ttl Seconds",
      "type": "number"
    },
    "heartbeat": {
      "$ref": "#/$defs/ClusterSharingHeartbeat",
      "description": "Configuration for background renewal of the cluster sharing lock while the Dagster step is running."
    }
  },
  "title": "ClusterSharing",
  "type": "object"
}

Fields:

Validators:

  • _warn_on_low_heartbeat_headroom

Attributes

enabled pydantic-field
enabled: bool = False

Whether to enable sharing of RayClusters.

match_dagster_labels pydantic-field
match_dagster_labels: MatchDagsterLabels

Configuration for matching on Dagster-generated labels.

match_labels pydantic-field
match_labels: dict[str, str] | None = None

Additional user-provided labels to match on.

ttl_seconds pydantic-field
ttl_seconds: float = DEFAULT_CLUSTER_SHARING_TTL_SECONDS

Time to live for the lock placed on the RayCluster resource, marking it as in use by the current Dagster step. The lock is renewed periodically while the step is running (see heartbeat), so this only needs to cover the gap between renewals.

heartbeat pydantic-field

Configuration for background renewal of the cluster sharing lock while the Dagster step is running.

dagster_ray.kuberay.configs.ClusterSharingHeartbeat pydantic-model

Bases: Config

Controls background renewal of the cluster sharing lock while the Dagster step is running.

Without renewal, the lock expires ttl_seconds after step start and the garbage collection sensor may delete the cluster while the step is still using it. With renewal, ttl_seconds can stay short: idle clusters are reaped promptly, active steps survive.

Each renewal updates the lock's heartbeat_at timestamp; created_at always points at the initial lock placement. A hanging step renews its lock indefinitely — set the dagster/max_runtime tag on runs to bound step runtime.

Show JSON schema:
{
  "description": "Controls background renewal of the cluster sharing lock while the Dagster step is running.\n\nWithout renewal, the lock expires `ttl_seconds` after step start and the garbage collection\nsensor may delete the cluster while the step is still using it. With renewal, `ttl_seconds`\ncan stay short: idle clusters are reaped promptly, active steps survive.\n\nEach renewal updates the lock's `heartbeat_at` timestamp; `created_at` always points at the\ninitial lock placement. A hanging step renews its lock indefinitely \u2014 set the\n`dagster/max_runtime` tag on runs to bound step runtime.",
  "properties": {
    "enabled": {
      "default": true,
      "description": "Whether to renew the cluster sharing lock in the background while the Dagster step is running.",
      "title": "Enabled",
      "type": "boolean"
    },
    "refresh_seconds": {
      "default": 10.0,
      "description": "How often to renew the lock. Must be well below `ClusterSharing.ttl_seconds` \u2014 if a renewal is missed, the lock must not have expired yet, or the cluster can be deleted mid-step.",
      "title": "Refresh Seconds",
      "type": "number"
    }
  },
  "title": "ClusterSharingHeartbeat",
  "type": "object"
}

Fields:

Attributes

enabled pydantic-field
enabled: bool = True

Whether to renew the cluster sharing lock in the background while the Dagster step is running.

refresh_seconds pydantic-field
refresh_seconds: float = 10.0

How often to renew the lock. Must be well below ClusterSharing.ttl_seconds — if a renewal is missed, the lock must not have expired yet, or the cluster can be deleted mid-step.


dagster_ray.kuberay.resources.base.BaseKubeRayResource pydantic-model

Bases: RayResource, ABC

Show JSON schema:
{
  "$defs": {
    "ExecutionOptionsConfig": {
      "properties": {
        "cpu": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Cpu"
        },
        "gpu": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Gpu"
        },
        "object_store_memory": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Object Store Memory"
        }
      },
      "title": "ExecutionOptionsConfig",
      "type": "object"
    },
    "Lifecycle": {
      "properties": {
        "create": {
          "default": true,
          "description": "Whether to create the resource. If set to `False`, the user can manually call `.create` instead.",
          "title": "Create",
          "type": "boolean"
        },
        "wait": {
          "default": true,
          "description": "Whether to wait for the remote Ray cluster to become ready to accept connections. If set to `False`, the user can manually call `.wait` instead.",
          "title": "Wait",
          "type": "boolean"
        },
        "connect": {
          "default": true,
          "description": "Whether to run `ray.init` against the remote Ray cluster. If set to `False`, the user can manually call `.connect` instead.",
          "title": "Connect",
          "type": "boolean"
        },
        "cleanup": {
          "default": "always",
          "description": "Resource cleanup policy. Determines when the resource should be deleted after Dagster step execution or during interruption.",
          "enum": [
            "never",
            "always",
            "on_exception"
          ],
          "title": "Cleanup",
          "type": "string"
        }
      },
      "title": "Lifecycle",
      "type": "object"
    },
    "RayDataExecutionOptions": {
      "properties": {
        "execution_options": {
          "$ref": "#/$defs/ExecutionOptionsConfig"
        },
        "cpu_limit": {
          "default": 5000,
          "title": "Cpu Limit",
          "type": "integer"
        },
        "gpu_limit": {
          "default": 0,
          "title": "Gpu Limit",
          "type": "integer"
        },
        "verbose_progress": {
          "default": true,
          "title": "Verbose Progress",
          "type": "boolean"
        },
        "use_polars": {
          "default": true,
          "title": "Use Polars",
          "type": "boolean"
        }
      },
      "title": "RayDataExecutionOptions",
      "type": "object"
    }
  },
  "properties": {
    "lifecycle": {
      "$ref": "#/$defs/Lifecycle",
      "description": "Actions to perform during resource setup."
    },
    "timeout": {
      "default": 600.0,
      "description": "Timeout for Ray readiness in seconds",
      "title": "Timeout",
      "type": "number"
    },
    "ray_init_options": {
      "additionalProperties": true,
      "description": "Additional keyword arguments to pass to `ray.init()` call, such as `runtime_env`, `num_cpus`, etc. Dagster's `EnvVar` is supported. More details in [Ray docs](https://docs.ray.io/en/latest/ray-core/api/doc/ray.init.html).",
      "title": "Ray Init Options",
      "type": "object"
    },
    "data_execution_options": {
      "$ref": "#/$defs/RayDataExecutionOptions"
    },
    "redis_port": {
      "default": 10001,
      "description": "Redis port for connection. Make sure to match with the actual available port.",
      "title": "Redis Port",
      "type": "integer"
    },
    "dashboard_port": {
      "default": 8265,
      "description": "Dashboard port for connection. Make sure to match with the actual available port.",
      "title": "Dashboard Port",
      "type": "integer"
    },
    "env_vars": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "description": "Environment variables to pass to the Ray cluster.",
      "title": "Env Vars"
    },
    "enable_tracing": {
      "default": false,
      "description": "Enable tracing: inject `RAY_PROFILING=1` and `RAY_task_events_report_interval_ms=0` into the Ray cluster configuration. This allows using `ray.timeline()` to fetch recorded task events. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-core/api/doc/ray.timeline.html#ray-timeline)",
      "title": "Enable Tracing",
      "type": "boolean"
    },
    "enable_actor_task_logging": {
      "default": false,
      "description": "Enable actor task logging: inject `RAY_ENABLE_RECORD_ACTOR_TASK_LOGGING=1` into the Ray cluster configuration.",
      "title": "Enable Actor Task Logging",
      "type": "boolean"
    },
    "enable_debug_post_mortem": {
      "default": false,
      "description": "Enable post-mortem debugging: inject `RAY_DEBUG_POST_MORTEM=1` into the Ray cluster configuration. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-observability/ray-distributed-debugger.html)",
      "title": "Enable Debug Post Mortem",
      "type": "boolean"
    },
    "enable_legacy_debugger": {
      "default": false,
      "description": "Enable legacy debugger: inject `RAY_DEBUG=legacy` into the Ray cluster configuration. Learn more: [KubeRay docs](https://docs.ray.io/en/latest/ray-observability/user-guides/debug-apps/ray-debugging.html#using-the-ray-debugger)",
      "title": "Enable Legacy Debugger",
      "type": "boolean"
    },
    "image": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Image to inject into the `RayCluster` spec. Defaults to `dagster/image` run tag. Images already provided in the `RayCluster` spec won't be overridden.",
      "title": "Image"
    },
    "deployment_name": {
      "default": "dev",
      "description": "Dagster deployment name. Is used as a prefix for the Kubernetes resource name. Dagster Cloud variables are used to determine the default value.",
      "title": "Deployment Name",
      "type": "string"
    },
    "failure_tolerance_timeout": {
      "default": 0.0,
      "description": "The period in seconds to wait for the cluster to transition out of `failed` state if it reaches it. This state can be transient under certain conditions. With the default value of 0, the first `failed` state appearance will raise an exception immediately.",
      "title": "Failure Tolerance Timeout",
      "type": "number"
    },
    "poll_interval": {
      "default": 1.0,
      "description": "Poll interval for various API requests",
      "title": "Poll Interval",
      "type": "number"
    }
  },
  "title": "BaseKubeRayResource",
  "type": "object"
}

Fields:

Attributes

image pydantic-field
image: str | None = None

Image to inject into the RayCluster spec. Defaults to dagster/image run tag. Images already provided in the RayCluster spec won't be overridden.

deployment_name pydantic-field
deployment_name: str = DEFAULT_DEPLOYMENT_NAME

Dagster deployment name. Is used as a prefix for the Kubernetes resource name. Dagster Cloud variables are used to determine the default value.

poll_interval pydantic-field
poll_interval: float = 1.0

Poll interval for various API requests

Functions

resolve_hostname
resolve_hostname(service_name: str, namespace: str) -> str

Resolve the host address from a Kubernetes service name and namespace.

By default, returns the in-cluster FQDN (e.g. my-svc.namespace.svc.cluster.local). Override this method to customize the host address, for example to use a custom domain.

The returned hostname is used to construct both the Ray Client address (ray://<host>:<redis_port>) and the dashboard URL (http://<host>:<dashboard_port>).

Parameters:

  • service_name (str) –

    The Kubernetes service name for the Ray head node.

  • namespace (str) –

    The Kubernetes namespace.

Returns:

  • str –

    The hostname to use for connecting to the Ray cluster.

Example
class MyKubeRayCluster(KubeRayCluster):
    def resolve_hostname(self, service_name: str, namespace: str) -> str:
        return f"{service_name}.{namespace}.company.com"
Source code in src/dagster_ray/kuberay/resources/base.py
def resolve_hostname(self, service_name: str, namespace: str) -> str:
    """Resolve the host address from a Kubernetes service name and namespace.

    By default, returns the in-cluster FQDN (e.g. `my-svc.namespace.svc.cluster.local`).
    Override this method to customize the host address, for example to use a custom domain.

    The returned hostname is used to construct both the Ray Client address (`ray://<host>:<redis_port>`)
    and the dashboard URL (`http://<host>:<dashboard_port>`).

    Args:
        service_name: The Kubernetes service name for the Ray head node.
        namespace: The Kubernetes namespace.

    Returns:
        The hostname to use for connecting to the Ray cluster.

    Example:
        ```python
        class MyKubeRayCluster(KubeRayCluster):
            def resolve_hostname(self, service_name: str, namespace: str) -> str:
                return f"{service_name}.{namespace}.company.com"
        ```
    """
    return k8s_service_fqdn(service_name, namespace)

Resources

dagster_ray.kuberay.KubeRayJobClientResource pydantic-model

Bases: ConfigurableResource[RayJobClient]

This configurable resource provides a dagster_ray.kuberay.client.RayJobClient.

Show JSON schema:
{
  "description": "This configurable resource provides a [dagster_ray.kuberay.client.RayJobClient][].",
  "properties": {
    "kube_context": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Kube Context"
    },
    "kube_config": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Kube Config"
    }
  },
  "title": "KubeRayJobClientResource",
  "type": "object"
}

Fields:

  • kube_context (str | None)
  • kube_config (str | None)

dagster_ray.kuberay.KubeRayClusterClientResource pydantic-model

Bases: ConfigurableResource[RayClusterClient]

This configurable resource provides a dagster_ray.kuberay.client.RayClusterClient.

Show JSON schema:
{
  "description": "This configurable resource provides a [dagster_ray.kuberay.client.RayClusterClient][].",
  "properties": {
    "kube_context": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Kube Context"
    },
    "kube_config": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Kube Config"
    }
  },
  "title": "KubeRayClusterClientResource",
  "type": "object"
}

Fields:

  • kube_context (str | None)
  • kube_config (str | None)

Sensors

dagster_ray.kuberay.sensors.cleanup_expired_kuberay_clusters

cleanup_expired_kuberay_clusters(
    context: SensorEvaluationContext, raycluster_client: ResourceParam[RayClusterClient]
) -> Generator[RunRequest | SkipReason, None, None]

A Dagster sensor that monitors shared RayCluster resources created by the current code location and submits jobs to delete clusters that either:

  • use Cluster Sharing (dagster/cluster-sharing=true) and have expired
  • are older than DAGSTER_RAY_CLUSTER_EXPIRATION_SECONDS (defaults to 4 hours)

A sharing lock expires ttl_seconds after its last heartbeat, so clusters with actively heartbeating locks are never deleted. Set the dagster/max_runtime tag on runs to prevent hanging steps from renewing their lock forever.

By default it monitors the ray namespace. This can be configured by setting DAGSTER_RAY_NAMESPACES (accepts a comma-separated list of namespaces).

Source code in src/dagster_ray/kuberay/sensors.py
@dg.sensor(job=delete_kuberay_clusters, minimum_interval_seconds=5 * 60)
def cleanup_expired_kuberay_clusters(
    context: dg.SensorEvaluationContext,
    raycluster_client: dg.ResourceParam[RayClusterClient],
) -> Generator[dg.RunRequest | dg.SkipReason, None, None]:
    """A Dagster sensor that monitors shared `RayCluster` resources created by the current code location and submits jobs to delete clusters that either:

    - use [Cluster Sharing](../tutorial/kuberay.md#cluster-sharing) (`dagster/cluster-sharing=true`) and have expired
    - are older than `DAGSTER_RAY_CLUSTER_EXPIRATION_SECONDS` (defaults to 4 hours)

    A sharing lock expires `ttl_seconds` after its last heartbeat, so clusters with actively heartbeating locks are never deleted.
    Set the `dagster/max_runtime` tag on runs to prevent hanging steps from renewing their lock forever.

    By default it monitors the `ray` namespace. This can be configured by setting `DAGSTER_RAY_NAMESPACES` (accepts a comma-separated list of namespaces).
    """
    assert context.code_location_origin is not None

    found_any = False
    namespaces = os.environ.get(DAGSTER_RAY_NAMESPACES_ENV_VAR, "ray").split(",")
    expiration_seconds = int(
        os.environ.get(
            DAGSTER_RAY_CLUSTER_EXPIRATION_SECONDS_ENV_VAR, DAGSTER_RAY_CLUSTER_EXPIRATION_SECONDS_DEFAULT_VALUE
        )
    )

    for namespace in namespaces:
        cluster_names = []
        for cluster in raycluster_client.list(
            namespace=namespace,
            label_selector=f"dagster/code-location={context.code_location_origin.location_name}",
        ).get("items", []):
            if cluster["metadata"].get("labels", {}).get("dagster/cluster-sharing") == "true":
                locks = ClusterSharingLock.parse_all_locks(
                    cast(dict[str, str], cluster.get("metadata", {}).get("annotations", {}))
                )
                alive_locks = ClusterSharingLock.get_alive_locks(locks)
                if not alive_locks:
                    context.log.info(
                        f"Found expired RayCluster with cluster sharing enabled: {cluster['metadata']['namespace']}/{cluster['metadata']['name']}"
                    )
                    cluster_names.append(cluster["metadata"]["name"])
            else:
                # check if the cluster age since creation time exceeds expiration_seconds

                # early exit for clusters with owners
                if cluster.get("metadata", {}).get("ownerReferences"):
                    continue

                if cluster.get("metadata", {}).get("creationTimestamp"):
                    cluster_age = datetime.now(timezone.utc) - datetime.strptime(
                        cluster["metadata"]["creationTimestamp"], "%Y-%m-%dT%H:%M:%SZ"
                    ).replace(tzinfo=timezone.utc)
                    if cluster_age.total_seconds() >= expiration_seconds:
                        context.log.info(
                            f"Found expired RayCluster (time since creation exceeds {expiration_seconds} seconds): {cluster['metadata']['namespace']}/{cluster['metadata']['name']}"
                        )
                        cluster_names.append(cluster["metadata"]["name"])

        if len(cluster_names) > 0:
            found_any = True
            yield dg.RunRequest(
                run_config=dg.RunConfig(
                    ops={
                        "delete_kuberay_clusters_op": DeleteKubeRayClustersConfig(
                            namespace=namespace,
                            clusters=[RayClusterRef(name=name) for name in cluster_names],
                        )
                    }
                )
            )

    if not found_any:
        yield dg.SkipReason(f"No expired RayClusters found in namespaces: {namespaces}")

A Dagster sensor that monitors shared RayCluster resources created by the current Dagster code location (with a dagster/code-location=<current-code-location> label selector) and submits jobs to delete clusters either: - use Cluster Sharing (dagster/cluster-sharing=true) and have expired - are older than DAGSTER_RAY_CLUSTER_EXPIRATION_SECONDS (defaults to 4 hours)

By default it monitors the ray namespace. This can be configured by setting DAGSTER_RAY_NAMESPACES (accepts a comma-separated list of namespaces).


Kubernetes API Clients

dagster_ray.kuberay.client.RayClusterClient

RayClusterClient(kube_config: str | None = None, kube_context: str | None = None, api_client: ApiClient | None = None)

Bases: BaseKubeRayClient[RayClusterStatus]

Source code in src/dagster_ray/kuberay/client/raycluster/client.py
def __init__(
    self,
    kube_config: str | None = None,
    kube_context: str | None = None,
    api_client: ApiClient | None = None,
) -> None:
    self.kube_config = kube_config
    self.kube_context = kube_context

    # note: this call must happen BEFORE creating the api clients
    if api_client is None:
        load_kubeconfig(context=self.kube_context, config_file=self.kube_config)

    super().__init__(group=GROUP, version=VERSION, kind=KIND, plural=PLURAL, api_client=api_client)

Functions

create
create(body: dict[str, Any], namespace: str) -> Any
Source code in src/dagster_ray/kuberay/client/base.py
def create(self, body: dict[str, Any], namespace: str) -> Any:
    return self._api.create_namespaced_custom_object(
        group=self.group,
        version=body.get("apiVersion", f"{self.group}/{self.version}").split("/")[1],
        plural=self.plural,
        body=body,
        namespace=namespace,
    )
delete
delete(name: str, namespace: str)
Source code in src/dagster_ray/kuberay/client/base.py
def delete(self, name: str, namespace: str):
    return self._api.delete_namespaced_custom_object(
        group=self.group,
        version=self.version,
        plural=self.plural,
        name=name,
        namespace=namespace,
    )
get
get(name: str, namespace: str) -> dict[str, Any]
Source code in src/dagster_ray/kuberay/client/base.py
def get(self, name: str, namespace: str) -> dict[str, Any]:
    from kubernetes.client import ApiException

    try:
        resource: Any = self._api.get_namespaced_custom_object(
            group=self.group,
            version=self.version,
            plural=self.plural,
            name=name,
            namespace=namespace,
        )
        return resource
    except ApiException as e:
        if e.status == 404:
            return {}
        raise
list
list(namespace: str, label_selector: str = '', async_req: bool = False) -> dict[str, Any]
Source code in src/dagster_ray/kuberay/client/base.py
def list(self, namespace: str, label_selector: str = "", async_req: bool = False) -> dict[str, Any]:
    from kubernetes.client import ApiException

    try:
        resource: Any = self._api.list_namespaced_custom_object(
            group=self.group,
            version=self.version,
            plural=self.plural,
            namespace=namespace,
            label_selector=label_selector,
            async_req=async_req,
        )
        if "items" in resource:
            return resource
        else:
            return {}
    except ApiException as e:
        if e.status == 404:
            return {}

        raise
update
update(name: str, namespace: str, body: Any)
Source code in src/dagster_ray/kuberay/client/base.py
def update(self, name: str, namespace: str, body: Any):
    return self._api.patch_namespaced_custom_object(
        group=self.group,
        version=self.version,
        plural=self.plural,
        name=name,
        body=body,
        namespace=namespace,
    )

dagster_ray.kuberay.client.RayJobClient

RayJobClient(kube_config: str | None = None, kube_context: str | None = None, api_client: ApiClient | None = None)

Bases: BaseKubeRayClient[RayJobStatus]

Source code in src/dagster_ray/kuberay/client/rayjob/client.py
def __init__(
    self,
    kube_config: str | None = None,
    kube_context: str | None = None,
    api_client: ApiClient | None = None,
) -> None:
    self.kube_config = kube_config
    self.kube_context = kube_context

    # this call must happen BEFORE creating K8s apis
    if api_client is None:
        load_kubeconfig(config_file=kube_config, context=kube_context)

    super().__init__(group=GROUP, version=VERSION, kind=KIND, plural=PLURAL, api_client=api_client)

Functions

create
create(body: dict[str, Any], namespace: str) -> Any
Source code in src/dagster_ray/kuberay/client/base.py
def create(self, body: dict[str, Any], namespace: str) -> Any:
    return self._api.create_namespaced_custom_object(
        group=self.group,
        version=body.get("apiVersion", f"{self.group}/{self.version}").split("/")[1],
        plural=self.plural,
        body=body,
        namespace=namespace,
    )
delete
delete(name: str, namespace: str)
Source code in src/dagster_ray/kuberay/client/base.py
def delete(self, name: str, namespace: str):
    return self._api.delete_namespaced_custom_object(
        group=self.group,
        version=self.version,
        plural=self.plural,
        name=name,
        namespace=namespace,
    )
get
get(name: str, namespace: str) -> dict[str, Any]
Source code in src/dagster_ray/kuberay/client/base.py
def get(self, name: str, namespace: str) -> dict[str, Any]:
    from kubernetes.client import ApiException

    try:
        resource: Any = self._api.get_namespaced_custom_object(
            group=self.group,
            version=self.version,
            plural=self.plural,
            name=name,
            namespace=namespace,
        )
        return resource
    except ApiException as e:
        if e.status == 404:
            return {}
        raise
list
list(namespace: str, label_selector: str = '', async_req: bool = False) -> dict[str, Any]
Source code in src/dagster_ray/kuberay/client/base.py
def list(self, namespace: str, label_selector: str = "", async_req: bool = False) -> dict[str, Any]:
    from kubernetes.client import ApiException

    try:
        resource: Any = self._api.list_namespaced_custom_object(
            group=self.group,
            version=self.version,
            plural=self.plural,
            namespace=namespace,
            label_selector=label_selector,
            async_req=async_req,
        )
        if "items" in resource:
            return resource
        else:
            return {}
    except ApiException as e:
        if e.status == 404:
            return {}

        raise
update
update(name: str, namespace: str, body: Any)
Source code in src/dagster_ray/kuberay/client/base.py
def update(self, name: str, namespace: str, body: Any):
    return self._api.patch_namespaced_custom_object(
        group=self.group,
        version=self.version,
        plural=self.plural,
        name=name,
        body=body,
        namespace=namespace,
    )