genai_output_cost

This page explains how to use the genai_output_cost function in APL.

The genai_output_cost function calculates the cost of output tokens (completion tokens) for a GenAI API call based on the model name and number of output tokens. This helps you understand and track the cost of generated responses separately from prompts.

You can use this function to analyze generation costs, optimize response length for cost efficiency, track output spending separately, or create detailed cost breakdowns.

For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

Splunk SPL users

In Splunk SPL, you would need to lookup pricing and calculate costs manually.

Splunk example

| lookup model_pricing model OUTPUT output_price
| eval output_cost=(output_tokens * output_price / 1000000)

ANSI SQL users

In ANSI SQL, you would join with a pricing table and calculate output costs.

SQL example

SELECT 
  l.*,
  (l.output_tokens * p.output_price / 1000000) as output_cost
FROM ai_logs l
JOIN model_pricing p ON l.model = p.model_name

Usage

Syntax

genai_output_cost(model, output_tokens)

Parameters

• model (string, required): The name of the AI model (for example, ‘gpt-4’, ‘claude-3-opus’, ‘gpt-3.5-turbo’).
• output_tokens (long, required): The number of output tokens (completion tokens) generated by the API call.

Returns

Returns a real number representing the cost in dollars (USD) for the output tokens based on the model’s pricing.

Use case examples

Log analysis

Analyze output costs to understand how much you spend on AI-generated responses versus input prompts.

Query

['sample-http-logs']
| where uri contains '/api/openai'
| extend model_name = tostring(todynamic(response_body)['model'])
| extend completion_tokens = tolong(todynamic(response_body)['usage']['completion_tokens'])
| extend response_cost = genai_output_cost(model_name, completion_tokens)
| summarize total_output_cost = sum(response_cost) by model_name, bin(_time, 1h)

Run in Playground

Output

_time	model_name	total_output_cost
2024-01-15T10:00:00Z	gpt-4	8.92
2024-01-15T10:00:00Z	gpt-3.5-turbo	0.45

This query breaks down output costs by model and time, showing where generation spending occurs.

List of related functions

• genai_input_cost: Calculates input token cost. Use this alongside output costs to understand the full cost breakdown.
• genai_cost: Calculates total cost (input + output). Use this when you need combined costs.
• genai_get_pricing: Gets pricing information. Use this to understand the pricing structure behind cost calculations.
• genai_extract_assistant_response: Extracts the response text. Combine with output costs to analyze cost per response.
• genai_is_truncated: Checks if responses were truncated. Use this to understand if token limits affected output costs.