Compare openai/gpt-4o-mini vs anthropic/claude-3.5-haiku

Model Comparison

Metric	gpt-4o-mini openai	claude-3.5-haiku anthropic
Pricing
Input Price	$0.15 / 1M tokens	$0.8 / 1M tokens
Output Price	$0.6 / 1M tokens	$4 / 1M tokens
Capabilities
Context Window	128K tokens	200K tokens
Capabilities	tools	tools
Input type	text, image	text, image
Category Scores
Overall Average	32.5	42.7
Academia	31.9	30.7
Finance	17.9	N/A
Marketing	42.5	49.8
Maths	14.7	N/A
Science	46.9	45.6
Writing	41.0	44.8
Benchmark Tests
AIME	11.7	3.3
AAII	21.2	20.2
AA Math Index	14.7	_
DROP	_	83.1
GPQA	42.6	41.2
HLE	4.0	3.5
HumanEval	_	88.1
LiveCodeBench	23.4	31.4
MATH-500	78.9	72.1
MMLU-Pro	64.8	63.4
SciCode	22.9	27.4

gpt-4o-mini Advantages

• Lower input token cost ($0.15 / 1M tokens vs $0.8 / 1M tokens)

claude-3.5-haiku Advantages

• Higher benchmark scores (42.7 vs 32.5)
• Larger context window (200K tokens vs 128K tokens)

Compare openai/gpt-4o-mini vs anthropic/claude-3.5-haiku