API Reference
Packages
vllm.ai/v1alpha1
Package v1alpha1 contains API Schema definitions for the vllm v1alpha1 API group
Resource Types
APIConfig
APIConfig defines API configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
batch_classification BatchClassificationConfig | Optional: {} |
AutoscalingSpec
AutoscalingSpec defines autoscaling configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled indicates if HPA is enabled | false | Optional: {} |
minReplicas integer | MinReplicas is the minimum number of replicas | 1 | Optional: {} |
maxReplicas integer | MaxReplicas is the maximum number of replicas | 10 | Optional: {} |
targetCPUUtilizationPercentage integer | TargetCPUUtilizationPercentage is the target CPU percentage | 80 | Optional: {} |
targetMemoryUtilizationPercentage integer | TargetMemoryUtilizationPercentage is the target memory percentage | Optional: {} |
BatchClassificationConfig
BatchClassificationConfig defines batch classification configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
max_batch_size integer | 100 | Optional: {} | |
concurrency_threshold integer | 5 | Optional: {} | |
max_concurrency integer | 8 | Optional: {} | |
metrics BatchMetricsConfig | Optional: {} |
BatchMetricsConfig
BatchMetricsConfig defines batch classification metrics configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | true | Optional: {} | |
detailed_goroutine_tracking boolean | true | Optional: {} | |
high_resolution_timing boolean | false | Optional: {} | |
sample_rate string | Sample rate for metrics (0.0-1.0). Stored as string to avoid float precision issues. | 1.0 | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} |
duration_buckets string array | Duration buckets for histograms. Stored as strings to avoid float precision issues. Example: ["0.001", "0.005", "0.01", "0.025", "0.05", "0.1", "0.25", "0.5", "1", "2.5", "5", "10", "30"] | Optional: {} | |
size_buckets integer array | Optional: {} |
CategoryModelConfig
CategoryModelConfig defines category model configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
model_id string | Optional: {} | ||
use_modernbert boolean | Optional: {} | ||
threshold string | Classification threshold (0.0-1.0). Stored as string to avoid float precision issues. | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} | |
use_cpu boolean | Optional: {} | ||
category_mapping_path string | Optional: {} |
ClassifierConfig
ClassifierConfig defines classifier configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
category_model CategoryModelConfig | Optional: {} | ||
pii_model PIIModelConfig | Optional: {} |
ComplexityCandidates
ComplexityCandidates defines candidate examples for complexity classification
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
candidates string array | List of candidate phrases or examples |
ComplexityRulesConfig
ComplexityRulesConfig defines complexity-based signal classification
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the complexity rule (e.g., "code-complexity", "reasoning-complexity") | ||
description string | Description of what this rule classifies | Optional: {} | |
threshold string | Threshold for difficulty classification (0.0-1.0). Stored as string to avoid float precision issues. Queries scoring above this threshold are classified as "hard" | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} | |
hard ComplexityCandidates | Hard candidates represent complex/difficult examples | ||
easy ComplexityCandidates | Easy candidates represent simple/easy examples | ||
composer RuleComposition | Composer allows filtering based on other signals (e.g., only apply this rule if domain:medical) | Optional: {} |
CompositionCondition
CompositionCondition defines a single composition condition
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Type of signal to check (e.g., "domain", "language", "category") | ||
name string | Name of the specific signal/rule value to match |
ConfigSpec
ConfigSpec defines the semantic router configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
routing JSON | Routing contains canonical v0.3 routing configuration under config.routing. It is intentionally preserved as an object so the operator can pass through the router-owned signal, projection, decision, and algorithm contract without lagging behind every router schema addition. | Type: object Optional: {} | |
embedding_models EmbeddingModelsConfig | Embedding models configuration (qwen3, gemma, mmbert) | Optional: {} | |
response_cache SemanticCacheConfig | Response cache configuration. | Optional: {} | |
semantic_cache SemanticCacheConfig | SemanticCache is the deprecated response-cache field. | Optional: {} | |
tools ToolsConfig | Tools configuration | Optional: {} | |
prompt_guard PromptGuardConfig | Prompt guard configuration | Optional: {} | |
classifier ClassifierConfig | Classifier configuration | Optional: {} | |
complexity_rules ComplexityRulesConfig array | Complexity rules for complexity-aware routing | Optional: {} | |
strategy string | Decision routing strategy ("priority" for priority-based matching) | Enum: [priority] Optional: {} | |
decisions DecisionConfig array | Routing decisions based on signals (domain, complexity, etc.) | Optional: {} | |
reasoning_families object (keys:string, values:ReasoningFamily) | Reasoning families | Optional: {} | |
default_reasoning_effort string | Default reasoning effort | Enum: [low medium high] Optional: {} | |
api APIConfig | API configuration | Optional: {} | |
observability ObservabilityConfig | Observability configuration | Optional: {} |
DecisionConfig
DecisionConfig defines a routing decision
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name is the unique identifier for this decision | ||
description string | Description provides information about what this decision handles | Optional: {} | |
priority integer | Priority is used for decision ordering - higher priority decisions are evaluated first | Optional: {} | |
rules RuleCombinationConfig | Rules defines the combination of conditions using AND/OR logic | ||
modelRefs ModelRefConfig array | ModelRefs contains model references for this decision | Optional: {} | |
preferred_endpoints string array | PreferredEndpoints specifies which vLLM endpoints to prefer for this decision | Optional: {} | |
plugins RawExtension array | Plugins contains policy configurations applied after rule matching | Optional: {} | |
algorithm JSON | Algorithm configures base model selection for this decision. It is preserved as a router-owned object so supported algorithms can evolve without requiring the operator CRD to duplicate every nested field. | Type: object Optional: {} |
EmbeddingEndpointConfig
EmbeddingEndpointConfig defines an external OpenAI-compatible embedding endpoint.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
base_url string | BaseURL is the base URL for the embedding endpoint, typically ending in /v1. | Optional: {} | |
model string | Model is the embedding model name sent to the external provider. | Optional: {} | |
api_key_env string | APIKeyEnv names the environment variable containing the provider API key. | Optional: {} | |
timeout_seconds integer | TimeoutSeconds is the request timeout for embedding calls. | Minimum: 0 Optional: {} | |
max_retries integer | MaxRetries is the maximum number of retry attempts for embedding calls. | Minimum: 0 Optional: {} | |
dimensions integer | Dimensions requests a provider-side output dimension when supported. | Minimum: 1 Optional: {} |
EmbeddingModelsConfig
EmbeddingModelsConfig defines configuration for embedding models
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
qwen3_model_path string | Path to Qwen3-Embedding-0.6B model directory Qwen3 provides 32K context and high quality embeddings (1024 dimensions) | Optional: {} | |
gemma_model_path string | Path to EmbeddingGemma-300M model directory Gemma provides 8K context and fast embeddings (768 dimensions) | Optional: {} | |
mmbert_model_path string | Path to mmBERT 2D Matryoshka embedding model directory Supports layer early exit (3/6/11/22) and dimension reduction (64-768) | Optional: {} | |
use_cpu boolean | Use CPU for inference (default: true) | true | Optional: {} |
embedding_config HNSWEmbeddingConfig | Embedding configuration for embedding-based classification | Optional: {} | |
endpoint EmbeddingEndpointConfig | Endpoint configures an external embedding provider endpoint. The API key should be injected into the semantic router pod environment and referenced by APIKeyEnv rather than stored directly in the CR. | Optional: {} |
ExporterConfig
ExporterConfig defines exporter configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | otlp | Optional: {} | |
endpoint string | jaeger:4317 | Optional: {} | |
insecure boolean | true | Optional: {} |
GatewayReference
GatewayReference references an existing Gateway
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the Gateway | MinLength: 1 | |
namespace string | Namespace of the Gateway | MinLength: 1 |
GatewaySpec
GatewaySpec defines Gateway API integration configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
existingRef GatewayReference | ExistingRef references an existing Gateway to use | Optional: {} |
HNSWCacheConfig
HNSWCacheConfig defines HNSW index configuration for hybrid/in-memory backends.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
use_hnsw boolean | UseHNSW enables HNSW indexing for faster similarity search | false | Optional: {} |
hnsw_m integer | M is the number of bi-directional links per node | 16 | Minimum: 2 Optional: {} |
hnsw_ef_construction integer | EfConstruction is the size of dynamic candidate list during construction | 200 | Minimum: 1 Optional: {} |
max_memory_entries integer | MaxMemoryEntries limits in-memory entries for hybrid backend | 1000 | Minimum: 0 Optional: {} |
HNSWEmbeddingConfig
HNSWEmbeddingConfig contains settings for embedding classification with HNSW indexing
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
backend string | Backend selects the embedding provider backend. | Enum: [candle openvino openai_compatible] Optional: {} | |
model_type string | ModelType specifies which embedding model to use Options: "qwen3" (1024-dim, 32K context), "gemma" (768-dim, 8K context), "mmbert" (64-768-dim, multilingual), "remote" (external provider) | Enum: [qwen3 gemma mmbert remote] Optional: {} | |
preload_embeddings boolean | PreloadEmbeddings enables precomputing candidate embeddings at startup | true | Optional: {} |
target_dimension integer | TargetDimension is the embedding dimension to use (default: 768) For mmBERT, supported local dimensions are 64, 128, 256, 512, 768. External providers may use other positive dimensions such as 1024, 1536, or 3072. | Minimum: 1 Optional: {} | |
target_layer integer | TargetLayer controls mmBERT early exit and is used only when ModelType is "mmbert". Lower layers reduce encoder work but may reduce quality; layer 22 uses the full encoder depth. Evaluate the latency and quality trade-off on representative deployment data. | Enum: [3 6 11 22] Optional: {} | |
enable_soft_matching boolean | EnableSoftMatching enables soft matching mode | true | Optional: {} |
min_score_threshold string | MinScoreThreshold for matching (0.0-1.0). Stored as string to avoid float precision issues. | 0.5 | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} |
ImageSpec
ImageSpec defines the container image configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
repository string | Repository is the container image repository | ghcr.io/vllm-project/semantic-router/extproc | Optional: {} |
tag string | Tag is the container image tag | latest | Optional: {} |
pullPolicy PullPolicy | PullPolicy is the image pull policy | IfNotPresent | Enum: [Always Never IfNotPresent] Optional: {} |
imageRegistry string | ImageRegistry is an optional registry prefix | Optional: {} |
IngressHost
IngressHost defines an ingress host
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
host string | Optional: {} | ||
paths IngressPath array | Optional: {} |
IngressPath
IngressPath defines an ingress path
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
path string | Optional: {} | ||
pathType string | Optional: {} | ||
servicePort integer | Optional: {} |
IngressSpec
IngressSpec defines ingress configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled indicates if ingress is enabled | false | Optional: {} |
className string | ClassName is the ingress class name | Optional: {} | |
annotations object (keys:string, values:string) | Annotations for ingress | Optional: {} | |
hosts IngressHost array | Hosts configuration | Optional: {} | |
tls IngressTLS array | TLS configuration | Optional: {} |
IngressTLS
IngressTLS defines ingress TLS configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
secretName string | Optional: {} | ||
hosts string array | Optional: {} |
LoRAAdapterSpec
LoRAAdapterSpec defines one LoRA adapter exposed by a VLLMEndpoint model.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name is the unique adapter identifier referenced by decision.modelRefs[].lora_name. | MaxLength: 100 MinLength: 1 | |
description string | Description provides a short human-readable summary for UI and docs surfaces. | MaxLength: 500 Optional: {} |
MetricsPortSpec
MetricsPortSpec extends PortSpec with enable flag
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
port integer | Port is the service port | Maximum: 65535 Minimum: 1 Optional: {} | |
targetPort integer | TargetPort is the container port | Maximum: 65535 Minimum: 1 Optional: {} | |
protocol Protocol | Protocol is the port protocol | TCP | Optional: {} |
enabled boolean | Enabled indicates if metrics should be exposed | true | Optional: {} |
MilvusCacheAuth
MilvusCacheAuth defines Milvus authentication.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether to use authentication | false | Optional: {} |
username string | Username for Milvus authentication | Optional: {} | |
password string | Password for Milvus authentication (plaintext - consider using PasswordSecretRef instead) | Optional: {} | |
password_secret_ref SecretKeySelector | PasswordSecretRef references a Secret containing the Milvus password Preferred over plaintext Password field for security | Optional: {} |
MilvusCacheBatch
MilvusCacheBatch defines batch operation settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
insert_batch_size integer | InsertBatchSize for bulk inserts | 100 | Minimum: 1 Optional: {} |
timeout integer | Timeout for batch operations in seconds | 60 | Minimum: 0 Optional: {} |
MilvusCacheCollection
MilvusCacheCollection defines Milvus collection configuration.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the Milvus collection | semantic_cache | Optional: {} |
description string | Description of the collection | Semantic cache for LLM request-response pairs | Optional: {} |
vector_field MilvusCacheVectorField | VectorField configuration for embeddings | Optional: {} | |
index MilvusCacheCollectionIndex | Index configuration for the collection | Optional: {} |
MilvusCacheCollectionIndex
MilvusCacheCollectionIndex defines collection index settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Type of index algorithm | HNSW | Enum: [HNSW IVF_FLAT IVF_SQ8 IVF_PQ] Optional: {} |
params MilvusCacheIndexParams | Params for the index | Optional: {} |
MilvusCacheCompaction
MilvusCacheCompaction defines compaction settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether auto-compaction is active | false | Optional: {} |
interval integer | Interval in seconds between compaction runs | 86400 | Minimum: 0 Optional: {} |
MilvusCacheConfig
MilvusCacheConfig defines Milvus cache backend configuration. Configure these settings when using Milvus as the semantic cache backend.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
connection MilvusCacheConnection | Connection settings for Milvus server | Optional: {} | |
collection MilvusCacheCollection | Collection settings for Milvus | Optional: {} | |
search MilvusCacheSearch | Search settings for Milvus queries | Optional: {} | |
performance MilvusCachePerformance | Performance tuning for Milvus | Optional: {} | |
data_management MilvusCacheDataManagement | DataManagement settings for TTL and compaction | Optional: {} | |
development MilvusCacheDevelopment | Development settings for Milvus cache | Optional: {} |
MilvusCacheConnection
MilvusCacheConnection defines Milvus connection parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
host string | Host is the Milvus server hostname or IP address | Optional: {} | |
port integer | Port is the Milvus server port | 19530 | Maximum: 65535 Minimum: 1 Optional: {} |
database string | Database name in Milvus | semantic_router_cache | Optional: {} |
timeout integer | Timeout for Milvus operations in seconds | 30 | Minimum: 0 Optional: {} |
auth MilvusCacheAuth | Auth configuration for Milvus authentication | Optional: {} | |
tls MilvusCacheTLS | TLS configuration for secure Milvus connections | Optional: {} |
MilvusCacheConnectionPool
MilvusCacheConnectionPool defines connection pool settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
max_connections integer | MaxConnections in the pool | 10 | Minimum: 1 Optional: {} |
max_idle_connections integer | MaxIdleConnections to keep | 5 | Minimum: 0 Optional: {} |
acquire_timeout integer | AcquireTimeout in seconds | 30 | Minimum: 0 Optional: {} |
MilvusCacheDataManagement
MilvusCacheDataManagement defines data lifecycle settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
ttl MilvusCacheTTL | TTL settings for automatic expiration | Optional: {} | |
compaction MilvusCacheCompaction | Compaction settings | Optional: {} |
MilvusCacheDevelopment
MilvusCacheDevelopment defines development-mode settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
drop_collection_on_startup boolean | DropCollectionOnStartup clears the collection when router starts (for testing) | false | Optional: {} |
auto_create_collection boolean | AutoCreateCollection automatically creates the collection if it doesn't exist | true | Optional: {} |
verbose_errors boolean | VerboseErrors includes detailed error messages in logs | true | Optional: {} |
MilvusCacheIndexParams
MilvusCacheIndexParams defines index parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
M integer | M is the number of bi-directional links for HNSW | 16 | Minimum: 2 Optional: {} |
efConstruction integer | EfConstruction for HNSW index building | 64 | Minimum: 1 Optional: {} |
MilvusCachePerformance
MilvusCachePerformance defines performance tuning.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
connection_pool MilvusCacheConnectionPool | ConnectionPool settings | Optional: {} | |
batch MilvusCacheBatch | Batch settings for operations | Optional: {} |
MilvusCacheSearch
MilvusCacheSearch defines Milvus search parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
params MilvusCacheSearchParams | Params for search operations | Optional: {} | |
topk integer | TopK is the number of results to return | 10 | Minimum: 1 Optional: {} |
consistency_level string | ConsistencyLevel for search operations Options: "Strong", "Session", "Bounded", "Eventually" | Session | Enum: [Strong Session Bounded Eventually] Optional: {} |
MilvusCacheSearchParams
MilvusCacheSearchParams defines search-time parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
ef integer | Ef is the search-time HNSW parameter | 64 | Minimum: 1 Optional: {} |
MilvusCacheTLS
MilvusCacheTLS defines TLS settings for Milvus connections.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether to use TLS | false | Optional: {} |
cert_file string | CertFile is the path to client certificate file | Optional: {} | |
key_file string | KeyFile is the path to client key file | Optional: {} | |
ca_file string | CAFile is the path to CA certificate file | Optional: {} |
MilvusCacheTTL
MilvusCacheTTL defines time-to-live settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether TTL is active | false | Optional: {} |
timestamp_field string | TimestampField is the field used for TTL calculation | created_at | Optional: {} |
cleanup_interval integer | CleanupInterval in seconds between cleanup runs | 3600 | Minimum: 0 Optional: {} |
MilvusCacheVectorField
MilvusCacheVectorField defines vector field configuration.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the vector field | embedding | Optional: {} |
dimension integer | Dimension of the embedding vectors | Minimum: 1 Optional: {} | |
metric_type string | MetricType for vector similarity Options: "IP" (inner product), "L2", "COSINE" | IP | Enum: [IP L2 COSINE] Optional: {} |
ModelRefConfig
ModelRefConfig defines a model reference for routing
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
model string | Model name to route to | ||
lora_name string | LoRAName is the optional LoRA adapter name | Optional: {} | |
use_reasoning boolean | UseReasoning enables reasoning mode for this model | Optional: {} | |
reasoning_effort string | ReasoningEffort specifies the reasoning effort level (low, medium, high) | Optional: {} |
ObservabilityConfig
ObservabilityConfig defines observability configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
tracing TracingConfig | Optional: {} |
OpenShiftFeaturesStatus
OpenShiftFeaturesStatus tracks OpenShift-specific feature status
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
routesEnabled boolean | RoutesEnabled indicates if OpenShift Routes are enabled | ||
routeHostname string | RouteHostname is the hostname of the created Route | Optional: {} |
OpenShiftSpec
OpenShiftSpec defines OpenShift-specific configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
routes RouteConfig | Routes configuration for OpenShift Routes | Optional: {} |
PIIModelConfig
PIIModelConfig defines PII model configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
model_id string | Optional: {} | ||
use_modernbert boolean | Optional: {} | ||
threshold string | Detection threshold (0.0-1.0). Stored as string to avoid float precision issues. | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} | |
use_cpu boolean | Optional: {} | ||
pii_mapping_path string | Optional: {} |
PersistenceSpec
PersistenceSpec defines persistence configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled indicates if persistence is enabled | true | Optional: {} |
storageClassName string | StorageClassName is the storage class name | standard | Optional: {} |
accessMode PersistentVolumeAccessMode | AccessMode is the access mode | ReadWriteOnce | Optional: {} |
size string | Size is the storage size | 10Gi | Optional: {} |
existingClaim string | ExistingClaim is an existing PVC to use | Optional: {} | |
annotations object (keys:string, values:string) | Annotations for the PVC | Optional: {} |
PortSpec
PortSpec defines a service port configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
port integer | Port is the service port | Maximum: 65535 Minimum: 1 Optional: {} | |
targetPort integer | TargetPort is the container port | Maximum: 65535 Minimum: 1 Optional: {} | |
protocol Protocol | Protocol is the port protocol | TCP | Optional: {} |
ProbeSpec
ProbeSpec defines probe configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled indicates if the probe is enabled | true | Optional: {} |
initialDelaySeconds integer | InitialDelaySeconds before probe starts | Optional: {} | |
periodSeconds integer | PeriodSeconds between probes | Optional: {} | |
timeoutSeconds integer | TimeoutSeconds for probe | Optional: {} | |
failureThreshold integer | FailureThreshold for probe | Optional: {} |
PromptGuardConfig
PromptGuardConfig defines prompt guard configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | true | Optional: {} | |
variant string | Variant selects a local Candle-backed model variant. It is mutually exclusive with Protocol. When both fields are omitted, the operator uses mmbert32k. | Enum: [candle mmbert32k] Optional: {} | |
protocol string | Protocol selects a remote HTTP backend's wire contract. Mutually exclusive with Variant. Requires an external model configured via a vllmEndpoints/externalModels entry with model_role="guardrail". | Enum: [http_chat http_classify] Optional: {} | |
model_id string | models/mmbert32k-jailbreak-detector-merged | Optional: {} | |
threshold string | Jailbreak detection threshold (0.0-1.0). Stored as string to avoid float precision issues. | 0.7 | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} |
use_cpu boolean | true | Optional: {} | |
jailbreak_mapping_path string | Optional: {} | ||
positive_labels string array | PositiveLabels lists the jailbreak_mapping labels that count as unsafe, for a custom backend whose positive class isn't named "jailbreak" (e.g. "INJECTION", "malicious"). Defaults to ["jailbreak"] when unset. | Optional: {} | |
on_error string | OnError selects what a prompt-guard classifier failure does to the rule that failed to evaluate. "allow" (the default) tolerates the failure and treats the content as not matching; "block" treats it as a positive detection, because an inference failure means the content could not be verified safe. Without this field on the CRD the setting is pruned by the API server and an operator-managed deployment silently fails open. | Enum: [allow block] Optional: {} |
QdrantCacheConfig
QdrantCacheConfig defines Qdrant cache backend configuration. Configure these settings when using Qdrant as the semantic cache backend.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
host string | Host is the Qdrant server hostname or IP address | Optional: {} | |
port integer | Port is the Qdrant gRPC port | 6334 | Maximum: 65535 Minimum: 1 Optional: {} |
api_key string | APIKey for Qdrant authentication | Optional: {} | |
use_tls boolean | UseTLS enables TLS for the Qdrant connection | false | Optional: {} |
collection_name string | CollectionName is the Qdrant collection to use for semantic cache | semantic_cache | Optional: {} |
connect_timeout integer | ConnectTimeout is the timeout in seconds for Qdrant connection | 10 | Optional: {} |
ReasoningFamily
ReasoningFamily defines reasoning family configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Optional: {} | ||
parameter string | Optional: {} |
RedisCacheConfig
RedisCacheConfig defines Redis cache backend configuration. Configure these settings when using Redis as the semantic cache backend.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
connection RedisCacheConnection | Connection settings for Redis server | Optional: {} | |
index RedisCacheIndex | Index settings for Redis vector search | Optional: {} | |
search RedisCacheSearch | Search settings for Redis queries | Optional: {} | |
development RedisCacheDevelopment | Development settings for Redis cache | Optional: {} |
RedisCacheConnection
RedisCacheConnection defines Redis connection parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
host string | Host is the Redis server hostname or IP address Example: "redis.default.svc.cluster.local" | Optional: {} | |
port integer | Port is the Redis server port | 6379 | Maximum: 65535 Minimum: 1 Optional: {} |
database integer | Database is the Redis database number to use | 0 | Minimum: 0 Optional: {} |
password string | Password for Redis authentication (plaintext - consider using PasswordSecretRef instead) | Optional: {} | |
password_secret_ref SecretKeySelector | PasswordSecretRef references a Secret containing the Redis password Preferred over plaintext Password field for security | Optional: {} | |
timeout integer | Timeout for Redis operations in seconds | 30 | Minimum: 0 Optional: {} |
tls RedisCacheTLS | TLS configuration for secure Redis connections | Optional: {} |
RedisCacheDevelopment
RedisCacheDevelopment defines development-mode settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
drop_index_on_startup boolean | DropIndexOnStartup clears the index when router starts (for testing) | false | Optional: {} |
auto_create_index boolean | AutoCreateIndex automatically creates the index if it doesn't exist | true | Optional: {} |
verbose_errors boolean | VerboseErrors includes detailed error messages in logs | true | Optional: {} |
RedisCacheIndex
RedisCacheIndex defines Redis vector index configuration.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the Redis index | semantic_cache_idx | Optional: {} |
prefix string | Prefix for Redis keys | doc: | Optional: {} |
vector_field RedisCacheVectorField | VectorField configuration for embeddings | Optional: {} | |
index_type string | IndexType specifies the index algorithm Options: "HNSW" (recommended), "FLAT" | HNSW | Enum: [HNSW FLAT] Optional: {} |
params RedisCacheIndexParams | Params for HNSW index | Optional: {} |
RedisCacheIndexParams
RedisCacheIndexParams defines HNSW index parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
M integer | M is the number of bi-directional links per node Higher values = better recall, more memory | 16 | Minimum: 2 Optional: {} |
efConstruction integer | EfConstruction is the size of dynamic candidate list during construction Higher values = better quality, slower indexing | 64 | Minimum: 1 Optional: {} |
RedisCacheSearch
RedisCacheSearch defines Redis search parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
topk integer | TopK is the number of results to return from vector search | 1 | Minimum: 1 Optional: {} |
RedisCacheTLS
RedisCacheTLS defines TLS settings for Redis connections.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether to use TLS for Redis connection | false | Optional: {} |
cert_file string | CertFile is the path to client certificate file | Optional: {} | |
key_file string | KeyFile is the path to client key file | Optional: {} | |
ca_file string | CAFile is the path to CA certificate file | Optional: {} |
RedisCacheVectorField
RedisCacheVectorField defines vector field configuration.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the vector field | embedding | Optional: {} |
dimension integer | Dimension of the embedding vectors For BERT: 384, for Qwen3: 1024, for Gemma: 768 | Minimum: 1 Optional: {} | |
metric_type string | MetricType for vector similarity Options: "COSINE", "IP" (inner product), "L2" (Euclidean) | COSINE | Enum: [COSINE IP L2] Optional: {} |
ResourceConfig
ResourceConfig defines resource configuration for tracing
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
service_name string | vllm-semantic-router | Optional: {} | |
service_version string | v0.1.0 | Optional: {} | |
deployment_environment string | development | Optional: {} |
RouteConfig
RouteConfig defines OpenShift Route configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled specifies whether to create an OpenShift Route | false | Optional: {} |
hostname string | Hostname for the Route (optional - OpenShift generates if empty) | Optional: {} | |
tls RouteTLSConfig | TLS configuration for the Route | Optional: {} |
RouteTLSConfig
RouteTLSConfig defines TLS configuration for OpenShift Routes
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
termination string | Termination type (edge, passthrough, reencrypt) | edge | Enum: [edge passthrough reencrypt] Optional: {} |
insecureEdgeTerminationPolicy string | InsecureEdgeTerminationPolicy for HTTP traffic | Redirect | Enum: [Allow Redirect None] Optional: {} |
RuleCombinationConfig
RuleCombinationConfig defines how to combine multiple rule conditions
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
operator string | Operator specifies how to combine conditions: "AND", "OR", or "NOT". NOT is strictly unary: it takes exactly one child condition and negates its result. Compose NOR/NAND by nesting NOT around OR/AND. | Enum: [AND OR NOT] | |
conditions RuleConditionConfig array | Conditions is the list of rule references to evaluate |
RuleComposition
RuleComposition defines how to compose/filter rules based on other signals
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
operator string | Operator for combining conditions (AND, OR, NOT). NOT is strictly unary and negates its single child. | Enum: [AND OR NOT] | |
conditions CompositionCondition array | List of conditions that must be met |
RuleConditionConfig
RuleConditionConfig references a specific rule by type and name
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Type specifies the signal or projection type referenced by this condition. | Enum: [keyword embedding domain fact_check user_feedback reask preference language context structure complexity modality authz jailbreak pii kb conversation event projection] | |
name string | Name is the name of the rule to reference |
SamplingConfig
SamplingConfig defines sampling configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | always_on | Optional: {} | |
rate string | Sampling rate (0.0-1.0). Stored as string to avoid float precision issues. | 1.0 | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} |
SemanticCacheConfig
SemanticCacheConfig defines semantic cache configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether semantic caching is active | true | Optional: {} |
backend_type string | BackendType specifies the cache backend to use Options: "memory" (default), "redis", "valkey", "milvus", "qdrant", "hybrid" | memory | Enum: [memory redis valkey milvus qdrant hybrid] Optional: {} |
similarity_threshold string | Similarity threshold for cache hits (0.0-1.0). Stored as string to avoid float precision issues. | 0.8 | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} |
max_entries integer | MaxEntries is the maximum number of cache entries (for memory/hybrid backends) | 1000 | Optional: {} |
ttl_seconds integer | TTLSeconds is the time-to-live for cache entries in seconds | 3600 | Optional: {} |
eviction_policy string | EvictionPolicy for in-memory cache ("fifo", "lru", "lfu") | fifo | Enum: [fifo lru lfu] Optional: {} |
redis RedisCacheConfig | Redis configuration (required when backend_type is "redis") | Optional: {} | |
valkey ValkeyCacheConfig | Valkey configuration (required when backend_type is "valkey") | Optional: {} | |
milvus MilvusCacheConfig | Milvus configuration (required when backend_type is "milvus") | Optional: {} | |
qdrant QdrantCacheConfig | Qdrant configuration (required when backend_type is "qdrant") | Optional: {} | |
embedding_model string | EmbeddingModel specifies which embedding model to use for semantic similarity Options: "mmbert" (default), "bert", "qwen3", "gemma" | mmbert | Enum: [bert qwen3 gemma mmbert] Optional: {} |
hnsw HNSWCacheConfig | HNSW configuration for hybrid/in-memory backends | Optional: {} |
SemanticRouter
SemanticRouter is the Schema for the semanticrouters API
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
apiVersion string | vllm.ai/v1alpha1 | ||
kind string | SemanticRouter | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata. | ||
spec SemanticRouterSpec | |||
status SemanticRouterStatus |
SemanticRouterList
SemanticRouterList contains a list of SemanticRouter
| Field | Description | Default | Validation |
|---|---|---|---|
apiVersion string | vllm.ai/v1alpha1 | ||
kind string | SemanticRouterList | ||
metadata ListMeta | Refer to Kubernetes API documentation for fields of metadata. | ||
items SemanticRouter array |
SemanticRouterSpec
SemanticRouterSpec defines the desired state of SemanticRouter
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
image ImageSpec | Image configuration | Optional: {} | |
replicas integer | Number of replicas | 1 | Minimum: 0 Optional: {} |
imagePullSecrets LocalObjectReference array | ImagePullSecrets for private registries | Optional: {} | |
serviceAccount ServiceAccountSpec | ServiceAccount configuration | Optional: {} | |
service ServiceSpec | Service configuration | Optional: {} | |
resources ResourceRequirements | Resource requirements | Optional: {} | |
persistence PersistenceSpec | Persistence configuration | Optional: {} | |
config ConfigSpec | Configuration overrides merged into the canonical v0.3 config.yaml. Router-wide runtime overrides land under config.global.router/services/stores/ integrations/model_catalog, with model-backed modules nested under config.global.model_catalog.modules. Provider defaults land under config.providers.defaults. | Optional: {} | |
toolsDb ToolEntry array | Tools database configuration | Optional: {} | |
vllmEndpoints VLLMEndpointSpec array | VLLMEndpoints is a Kubernetes-native backend discovery adapter. It generates canonical config.providers.models[].backend_refs and config.routing.modelCards entries. | Optional: {} | |
autoscaling AutoscalingSpec | Autoscaling configuration | Optional: {} | |
startupProbe ProbeSpec | Probes configuration | Optional: {} | |
livenessProbe ProbeSpec | Optional: {} | ||
readinessProbe ProbeSpec | Optional: {} | ||
securityContext SecurityContext | Security context | Optional: {} | |
podSecurityContext PodSecurityContext | Pod security context | Optional: {} | |
podAnnotations object (keys:string, values:string) | Pod annotations | Optional: {} | |
nodeSelector object (keys:string, values:string) | Node selector | Optional: {} | |
tolerations Toleration array | Tolerations | Optional: {} | |
affinity Affinity | Affinity | Optional: {} | |
env EnvVar array | Environment variables | Optional: {} | |
args string array | Container arguments | Optional: {} | |
gateway GatewaySpec | Gateway integration for reusing existing gateways | Optional: {} | |
openshift OpenShiftSpec | OpenShift-specific features | Optional: {} | |
ingress IngressSpec | Ingress configuration | Optional: {} |
SemanticRouterStatus
SemanticRouterStatus defines the observed state of SemanticRouter
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
conditions Condition array | Conditions represent the latest available observations of the SemanticRouter's state | Optional: {} | |
observedGeneration integer | ObservedGeneration reflects the generation of the most recently observed SemanticRouter | Optional: {} | |
replicas integer | Replicas is the current number of replicas | Optional: {} | |
readyReplicas integer | ReadyReplicas is the number of ready replicas | Optional: {} | |
phase string | Phase represents the current phase of the SemanticRouter | Optional: {} | |
gatewayMode string | GatewayMode indicates deployment mode: standalone or gateway-integration | Optional: {} | |
openshiftFeatures OpenShiftFeaturesStatus | OpenShiftFeatures tracks OpenShift-specific feature status | Optional: {} |
ServiceAccountSpec
ServiceAccountSpec defines service account configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
create boolean | Create specifies whether to create a service account | true | Optional: {} |
name string | Name of the service account to use | Optional: {} | |
annotations object (keys:string, values:string) | Annotations for the service account | Optional: {} |
ServiceBackend
ServiceBackend defines a direct Kubernetes service backend
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Service name | MinLength: 1 | |
namespace string | Service namespace (defaults to same namespace) | Optional: {} | |
port integer | Service port | Minimum: 1 |
ServiceSpec
ServiceSpec defines the service configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type ServiceType | Type is the service type | ClusterIP | Enum: [ClusterIP NodePort LoadBalancer] Optional: {} |
grpc PortSpec | GRPC port configuration | Optional: {} | |
api PortSpec | API port configuration | Optional: {} | |
metrics MetricsPortSpec | Metrics port configuration | Optional: {} |
Tool
Tool defines a tool function
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Enum: [function] Optional: {} | ||
function ToolFunction | Optional: {} |
ToolEntry
ToolEntry defines a tool entry in the tools database
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
tool Tool | Optional: {} | ||
description string | Optional: {} | ||
category string | Optional: {} | ||
tags string array | Optional: {} |
ToolFunction
ToolFunction defines a tool function details
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Optional: {} | ||
description string | Optional: {} | ||
parameters ToolParameters | Optional: {} |
ToolParameters
ToolParameters defines tool function parameters
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Optional: {} | ||
properties JSON | Type: object Optional: {} | ||
required string array | Optional: {} |
ToolsConfig
ToolsConfig defines tools configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | true | Optional: {} | |
top_k integer | 3 | Optional: {} | |
similarity_threshold string | Similarity threshold for tool selection (0.0-1.0). Stored as string to avoid float precision issues. | 0.2 | Pattern: ^0(\.[0-9]+)?$|^1(\.0+)?$ Optional: {} |
tools_db_path string | config/tools_db.json | Optional: {} | |
fallback_to_empty boolean | true | Optional: {} |
TracingConfig
TracingConfig defines tracing configuration
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | false | Optional: {} | |
provider string | opentelemetry | Optional: {} | |
exporter ExporterConfig | Optional: {} | ||
sampling SamplingConfig | Optional: {} | ||
resource ResourceConfig | Optional: {} |
VLLMBackend
VLLMBackend specifies how to reach the vLLM service
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
type string | Type of backend: kserve, llamastack, or service | Enum: [kserve llamastack service] | |
inferenceServiceName string | For type=kserve: InferenceService name for auto-discovery | Optional: {} | |
discoveryLabels object (keys:string, values:string) | For type=llamastack: Labels to match services | Optional: {} | |
service ServiceBackend | For type=service: Direct service configuration | Optional: {} |
VLLMEndpointSpec
VLLMEndpointSpec defines a vLLM model backend endpoint
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the backend ref generated under config.providers.models[].backend_refs | MinLength: 1 | |
model string | Model name as reported by vLLM (e.g., "Model-A", "llama3-8b") | MinLength: 1 | |
reasoningFamily string | Reasoning family for the model (e.g., "qwen3", "deepseek", "gpt") | Optional: {} | |
loras LoRAAdapterSpec array | LoRAs declares the LoRA adapters exposed for this logical model in routing.modelCards. | MaxItems: 50 Optional: {} | |
backend VLLMBackend | Backend configuration | ||
weight integer | Weight for load balancing (default: 1) | 1 | Optional: {} |
ValkeyCacheConfig
ValkeyCacheConfig defines Valkey cache backend configuration. Configure these settings when using Valkey as the semantic cache backend.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
connection ValkeyCacheConnection | Connection settings for Valkey server | Optional: {} | |
index ValkeyCacheIndex | Index settings for Valkey vector search | Optional: {} | |
search ValkeyCacheSearch | Search settings for Valkey queries | Optional: {} | |
development ValkeyCacheDevelopment | Development settings for Valkey cache | Optional: {} |
ValkeyCacheConnection
ValkeyCacheConnection defines Valkey connection parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
host string | Host is the Valkey server hostname or IP address Example: "valkey.default.svc.cluster.local" | Optional: {} | |
port integer | Port is the Valkey server port | 6379 | Maximum: 65535 Minimum: 1 Optional: {} |
database integer | Database is the Valkey database number to use | 0 | Minimum: 0 Optional: {} |
password string | Password for Valkey authentication (plaintext - consider using PasswordSecretRef instead) | Optional: {} | |
password_secret_ref SecretKeySelector | PasswordSecretRef references a Secret containing the Valkey password Preferred over plaintext Password field for security | Optional: {} | |
timeout integer | Timeout for Valkey operations in seconds | 30 | Minimum: 0 Optional: {} |
tls ValkeyCacheTLS | TLS configuration for secure Valkey connections | Optional: {} |
ValkeyCacheDevelopment
ValkeyCacheDevelopment defines development-mode settings.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
drop_index_on_startup boolean | DropIndexOnStartup clears the index when router starts (for testing) | false | Optional: {} |
auto_create_index boolean | AutoCreateIndex automatically creates the index if it doesn't exist | true | Optional: {} |
verbose_errors boolean | VerboseErrors includes detailed error messages in logs | true | Optional: {} |
ValkeyCacheIndex
ValkeyCacheIndex defines Valkey vector index configuration.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the Valkey index | semantic_cache_idx | Optional: {} |
prefix string | Prefix for Valkey keys | doc: | Optional: {} |
vector_field ValkeyCacheVectorField | VectorField configuration for embeddings | Optional: {} | |
index_type string | IndexType specifies the index algorithm Options: "HNSW" (recommended), "FLAT" | HNSW | Enum: [HNSW FLAT] Optional: {} |
params ValkeyCacheIndexParams | Params for HNSW index | Optional: {} |
ValkeyCacheIndexParams
ValkeyCacheIndexParams defines HNSW index parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
M integer | M is the number of bi-directional links per node Higher values = better recall, more memory | 16 | Minimum: 2 Optional: {} |
efConstruction integer | EfConstruction is the size of dynamic candidate list during construction Higher values = better quality, slower indexing | 64 | Minimum: 1 Optional: {} |
ValkeyCacheSearch
ValkeyCacheSearch defines Valkey search parameters.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
topk integer | TopK is the number of results to return from vector search | 1 | Minimum: 1 Optional: {} |
ValkeyCacheTLS
ValkeyCacheTLS defines TLS settings for Valkey connections.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean | Enabled controls whether to use TLS for Valkey connection | false | Optional: {} |
cert_file string | CertFile is the path to client certificate file | Optional: {} | |
key_file string | KeyFile is the path to client key file | Optional: {} | |
ca_file string | CAFile is the path to CA certificate file | Optional: {} |
ValkeyCacheVectorField
ValkeyCacheVectorField defines vector field configuration.
Appears in:
| Field | Description | Default | Validation |
|---|---|---|---|
name string | Name of the vector field | embedding | Optional: {} |
dimension integer | Dimension of the embedding vectors For BERT: 384, for Qwen3: 1024, for Gemma: 768 | Minimum: 1 Optional: {} | |
metric_type string | MetricType for vector similarity Options: "COSINE", "IP" (inner product), "L2" (Euclidean) | COSINE | Enum: [COSINE IP L2] Optional: {} |