Go SDKAPI ReferenceModels

PreferredMinThroughput - Go SDK

PreferredMinThroughput type definition

The Go SDK and docs are currently in beta. Report issues on GitHub.

Preferred minimum throughput (in tokens per second). Can be a number (applies to p50) or an object with percentile-specific cutoffs. Endpoints below the threshold(s) may still be used, but are deprioritized in routing. When using fallback models, this may cause a fallback model to be used instead of the primary model if it meets the threshold.

Supported Types

1preferredMinThroughput := components.CreatePreferredMinThroughputNumber(float64{/* values here */})

PercentileThroughputCutoffs

1preferredMinThroughput := components.CreatePreferredMinThroughputPercentileThroughputCutoffs(components.PercentileThroughputCutoffs{/* values here */})

1preferredMinThroughput := components.CreatePreferredMinThroughputAny(any{/* values here */})

Union Discrimination

Use the Type field to determine which variant is active, then access the corresponding field:

1switch preferredMinThroughput.Type {
2	case components.PreferredMinThroughputTypeNumber:
3		// preferredMinThroughput.Number is populated
4	case components.PreferredMinThroughputTypePercentileThroughputCutoffs:
5		// preferredMinThroughput.PercentileThroughputCutoffs is populated
6	case components.PreferredMinThroughputTypeAny:
7		// preferredMinThroughput.Any is populated
8}