Together AI
AI cloud for running, fine-tuning, and deploying open-source models via serverless inference and on-demand GPU clusters
USAGE-BASED Verified JUN 24, 2026
Groq
Ultra-fast, low-cost inference for open models on custom LPU chips (groq.com — not xAI's Grok)
Groq runs open models — Llama, GPT-OSS, Qwen — on its custom LPU inference chip, delivering hundreds of tokens per second at low cost. GroqCloud has a genuine free tier (no credit card) and pay-as-you-go per-token pricing: Llama 3.1 8B runs about $0.05 per 1M input tokens at ~840 tokens/second. Best for developers who need fast, cheap inference via an OpenAI-compatible API. Note: this is Groq, not Elon Musk's xAI Grok.
Verified JUN 24, 2026 USAGE-BASED Live
What is Groq?
Groq is an inference company built around a custom chip it calls the LPU, or Language Processing Unit, designed specifically to run language models fast. Its cloud service, GroqCloud, hosts open models — Llama 3.1, 3.3, and 4, GPT-OSS, Qwen3, and speech models like Whisper and Orpheus — and serves them at hundreds to over a thousand tokens per second. The headline trait is speed: where general-purpose GPUs stream tokens at a steady pace, Groq’s hardware generates them fast enough to make real-time chat and voice feel instant. (Note the spelling: this is Groq, not xAI’s similarly named Grok chatbot.)
Access is through an OpenAI-compatible API, so existing code can switch over with a roughly two-line change. Pricing is usage-based and varies by model — Llama 3.1 8B runs about $0.05 per 1M input tokens at around 840 tokens per second, while larger models cost more. GroqCloud has a genuine free tier with no credit card required, gated by rate limits, and a pay-as-you-go Developer tier with up to ~10x higher limits, a 50%-off Batch API, and 50%-off prompt caching. An Enterprise tier adds on-prem deployment and custom models.
Who is it for?
Groq is a developer platform for applications where inference speed and cost matter more than access to closed frontier models. If your product lives or dies on latency — voice, live chat, fast agents — Groq’s throughput is its main draw.
- Developers building real-time apps — conversational assistants, voice agents, live tools — that need low latency.
- Teams running agentic or RAG pipelines where fast token generation compounds across many model calls.
- Cost-sensitive builders processing large batches who benefit from low per-token rates and discounted batch jobs.
- Engineers already on the OpenAI API who want to test a faster, cheaper open-model backend with minimal code changes.
How much does Groq cost?
Starting price: $0 · Free tier: yes · Model: usage-based
Pricing verified JUN 24, 2026
Price history tracked from June 2026
| Plan | Price | Includes |
|---|---|---|
| Free | Free | All models, no credit card required · Free API key via the console · Rate-limited (e.g. ~30 requests/min) · Good for prototyping and testing |
| Developer | Pay per token | Up to ~10x higher rate limits · Tokens-as-a-service, on demand · Batch API at 50% off async jobs · Prompt caching at 50% off cached input |
| Enterprise | Custom | On-prem deployment options · Custom and fine-tuned models · Access to premium models · Dedicated support — contact sales |
What are Groq's key features?
- Custom LPU (Language Processing Unit) silicon purpose-built for inference
- OpenAI-compatible API for near drop-in migration
- Hosts open models including Llama 3.1/3.3/4, GPT-OSS, and Qwen3
- Speech: Whisper Large v3 Turbo transcription and Orpheus text-to-speech
- Prompt caching (50% off cached input) and a Batch API (50% off async)
- Free web console and playground with a free API key
What people use Groq for
- 01 Real-time, low-latency chatbots and conversational assistants
- 02 High-throughput agentic and RAG pipelines that need fast token generation
- 03 Cost-sensitive large-scale batch inference jobs
- 04 Speech-to-text and text-to-speech applications
Pros and cons
| Pros | Cons |
|---|---|
| Among the fastest inference available — hundreds to 1,000+ tokens per second per model | Limited to open and hosted models — no closed frontier models like GPT-4 or Claude class |
| Very low, transparent per-token pricing with a genuine free tier (no credit card) | Free-tier rate limits are tight for production workloads |
| OpenAI-compatible API makes migration trivial | Per-token rates and the model lineup vary significantly and change over time |
What are the best Groq alternatives?
OpenRouter
Unified API to 400+ LLMs from 70+ providers through one OpenAI-compatible endpoint, with automatic failover and pass-through token pricing
USAGE-BASED Verified JUN 23, 2026
Replicate
Run and fine-tune thousands of open-source AI models with one line of code via a cloud API, billed per second of GPU or CPU compute
USAGE-BASED Verified JUN 23, 2026
Ollama
Free, open-source tool to run open-weight LLMs locally via CLI, desktop, or API — with an optional paid cloud for larger models
FREEMIUM Verified JUN 23, 2026
Head-to-head comparisons
How people make money with Groq
- Build latency-sensitive AI products (live chat assistants, voice apps, real-time agents) on Groq's fast inference and charge a subscription, keeping the margin over the low per-token cost you pay
- Offer high-throughput batch processing services — summarization, classification, enrichment of large datasets — using Groq's discounted Batch API and billing clients per record or per project
Frequently asked questions
Is Groq the same as Grok?
No. Groq (groq.com) is an inference company that runs open models on its custom LPU chip via GroqCloud. Grok is a separate chatbot from Elon Musk's xAI. The names are similar but the products and companies are unrelated.
Is Groq free to use?
Groq has a genuine free tier on GroqCloud that requires no credit card and gives you an API key, subject to rate limits such as around 30 requests per minute. Beyond that, the Developer tier is pay-as-you-go per token with higher limits.
How fast is Groq?
Groq's LPU hardware delivers very high throughput — hundreds to over 1,000 tokens per second depending on the model. For example, Llama 3.1 8B runs around 840 tokens per second, which makes it well suited to real-time chat and voice applications.
How much does Groq cost?
GroqCloud is usage-based and varies by model. Llama 3.1 8B is about $0.05 per 1M input tokens and $0.08 output, while Llama 3.3 70B is roughly $0.59 input and $0.79 output per 1M. Batch jobs and cached prompts get 50% discounts.
Which models does Groq support?
Groq hosts open models including Llama 3.1, 3.3, and 4, GPT-OSS 20B and 120B, and Qwen3, plus speech models like Whisper Large v3 Turbo and Orpheus text-to-speech. The lineup changes over time, so check the pricing page for current models.
Does Groq work with existing OpenAI code?
Yes. Groq provides an OpenAI-compatible API, so applications written for the OpenAI SDK can switch to Groq with roughly a two-line change to the base URL and key. This makes it easy to test Groq's speed against your current provider.