Skip to main content

AI__TOP_K

relay_conventions::attributes

Constant AI__TOP_K 

Source 
pub const AI__TOP_K: &str = "ai.top_k";
👎Deprecated:

Use GEN_AI__REQUEST__TOP_K (gen_ai.request.top_k) instead.

Expand description

Limits the model to only consider the K most likely next tokens, where K is an integer (e.g., top_k=20 means only the 20 highest probability tokens are considered).

  • PII: Maybe
  • Rewriting behavior: Backfill

§Aliases