gRPC — Protobuf over HTTP/2

What it is#

gRPC is a high-performance RPC framework open-sourced by Google in 2015. It’s a descendant of Google’s internal Stubby framework, which has carried roughly all of Google’s service-to-service traffic for over a decade. The shape of gRPC:

• Service definition first. You write a .proto file declaring the service, its methods, and the request and response messages. Protobuf is the IDL.
• Code generation. A protoc plugin reads the .proto file and emits a client stub and server skeleton in your language of choice. There are official plugins for C++, Java, Python, Go, Ruby, C#, Node.js, Objective-C, PHP, Dart, Kotlin, and Swift.
• HTTP/2 transport. gRPC runs over HTTP/2 because it needs HTTP/2’s multiplexing (many concurrent calls over one connection), bidirectional streams, and header compression.
• Four streaming modes. Unary, server-stream, client-stream, bidi-stream. Streaming is first-class, not bolted on.
• A status-code vocabulary. gRPC defines its own status taxonomy (16 codes from OK to UNAUTHENTICATED) — not HTTP status codes.

The result: an internal call looks like an in-process function call. client.GetUser(req) over the wire, with the same syntax as getUser(req) in your local code.

When to use it#

Reach for gRPC when:

• The consumer is another internal service, especially in a polyglot environment. The Python frontend calling the Go backend calling the Java pricing engine — gRPC’s codegen makes the cross-language call ergonomic.
• Throughput matters. Binary Protobuf is 3-10x smaller than JSON. HTTP/2 multiplexes many calls over one TCP connection, eliminating REST’s connection-pool sprawl.
• The contract is the source of truth. A .proto file is the canonical contract; client and server are generated from it. No drift between docs and reality.
• You need streaming. Server-side log tails, client-side telemetry upload, bidirectional chat — gRPC’s four streaming modes cover all of them.
• You want deadlines, cancellation, metadata, and back-pressure as first-class concepts rather than HTTP-header conventions.

Avoid gRPC (or be careful) when:

• The consumer is a browser. Browsers cannot speak raw HTTP/2 frames the way gRPC needs. You need grpc-web with a translating proxy (Envoy, Connect), which complicates the deployment. Many teams keep a REST or GraphQL surface for browsers and gRPC internally.
• The consumer is a non-technical partner. “Run protoc to generate a stub” is more friction than “curl the endpoint.” For public partner APIs, REST has lower onboarding cost.
• Debugging with curl matters. A gRPC call cannot be made with curl alone. grpcurl, BloomRPC, and Postman have gRPC support, but the tooling lag is real.
• The team has no Protobuf experience. Protobuf has its own pitfalls (field numbering, reserved, oneof, evolution rules) and ramp-up takes a sprint.

How it works#

Protobuf as the IDL#

A gRPC service starts with a .proto file. The same one defines messages, RPC methods, and (optionally) field-level validation rules.

syntax = "proto3";

package example.orders.v1;

option go_package = "github.com/example/orders/v1;ordersv1";

import "google/protobuf/timestamp.proto";

service OrderService {
  // Unary: one request, one response.
  rpc GetOrder(GetOrderRequest) returns (Order);

  // Server streaming: one request, stream of responses.
  rpc StreamOrderUpdates(StreamOrderUpdatesRequest) returns (stream OrderUpdate);

  // Client streaming: stream of requests, one response.
  rpc UploadOrderEvents(stream OrderEvent) returns (UploadAck);

  // Bidirectional streaming: stream of requests, stream of responses.
  rpc OrderChat(stream ChatMessage) returns (stream ChatMessage);
}

message GetOrderRequest {
  string order_id = 1;
}

message Order {
  string id = 1;
  OrderStatus status = 2;
  Money total = 3;
  repeated OrderItem items = 4;
  google.protobuf.Timestamp created_at = 5;
}

enum OrderStatus {
  ORDER_STATUS_UNSPECIFIED = 0;
  ORDER_STATUS_PENDING = 1;
  ORDER_STATUS_CONFIRMED = 2;
  ORDER_STATUS_SHIPPED = 3;
  ORDER_STATUS_DELIVERED = 4;
  ORDER_STATUS_CANCELLED = 5;
}

message Money {
  string currency = 1;
  int64 value_minor = 2;
}

message OrderItem {
  string sku = 1;
  int32 quantity = 2;
}

message StreamOrderUpdatesRequest { string customer_id = 1; }
message OrderUpdate { Order order = 1; }
message OrderEvent { string event_type = 1; string order_id = 2; }
message UploadAck { int32 accepted = 1; }
message ChatMessage { string from = 1; string body = 2; }

Things worth knowing about Protobuf:

• Field numbers are permanent. Once a field has number 3, it has number 3 forever. Renumbering breaks every deployed binary. Use reserved 3; if you remove a field.
• proto3 makes scalar fields default to zero values. No optional/required distinction (mostly — optional was reintroduced in proto3.15). Absent and zero are indistinguishable for scalars unless you use optional.
• Backward-compatible evolution. Adding new fields is safe (old clients skip them). Removing fields is safe if you reserved the number. Changing a field’s type is not safe.
• oneof models a union — exactly one of N fields is set. Useful for polymorphic responses.

Code generation#

protoc --go_out=. --go-grpc_out=. orders.proto emits Go code. Equivalent invocations exist for every supported language. The generated code:

• Server skeleton — an interface you implement (type OrderServiceServer interface { GetOrder(ctx, *GetOrderRequest) (*Order, error); ... }).
• Client stub — a typed object with one method per RPC (client.GetOrder(ctx, &GetOrderRequest{OrderId: "ord_..."})).
• Message structs — strongly-typed types for every message in the proto.

In Go, Python, Java, and C#, the generated code is idiomatic for the language. The polyglot codegen story is what makes gRPC viable across heterogeneous backend teams.

A unary call across languages#

The same GetOrder call from a Python, Go, and Node client. The Protobuf wire format is identical; the language is style.

import grpc
from example.orders.v1 import orders_pb2, orders_pb2_grpc

channel = grpc.secure_channel(
    "orders.internal:443",
    grpc.ssl_channel_credentials(),
)
client = orders_pb2_grpc.OrderServiceStub(channel)

metadata = (("authorization", "Bearer eyJhbGciOi..."),)
req = orders_pb2.GetOrderRequest(order_id="ord_a3f9c2")

try:
    order = client.GetOrder(req, timeout=2.0, metadata=metadata)
    print(order.id, orders_pb2.OrderStatus.Name(order.status))
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.NOT_FOUND:
        print("no such order")
    elif e.code() == grpc.StatusCode.DEADLINE_EXCEEDED:
        print("timed out — retry")
    else:
        raise

package main

import (
    "context"
    "crypto/tls"
    "fmt"
    "time"

    ordersv1 "github.com/example/orders/v1"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials"
    "google.golang.org/grpc/metadata"
    "google.golang.org/grpc/status"
    "google.golang.org/grpc/codes"
)

func main() {
    creds := credentials.NewTLS(&tls.Config{})
    conn, err := grpc.Dial("orders.internal:443", grpc.WithTransportCredentials(creds))
    if err != nil { panic(err) }
    defer conn.Close()

    client := ordersv1.NewOrderServiceClient(conn)

    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()
    ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer eyJhbGciOi...")

    order, err := client.GetOrder(ctx, &ordersv1.GetOrderRequest{OrderId: "ord_a3f9c2"})
    if err != nil {
        if status.Code(err) == codes.NotFound {
            fmt.Println("no such order")
            return
        }
        panic(err)
    }
    fmt.Println(order.Id, order.Status)
}

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");

const packageDef = protoLoader.loadSync("orders.proto");
const proto = grpc.loadPackageDefinition(packageDef).example.orders.v1;

const creds = grpc.credentials.createSsl();
const client = new proto.OrderService("orders.internal:443", creds);

const meta = new grpc.Metadata();
meta.add("authorization", "Bearer eyJhbGciOi...");

const deadline = new Date(Date.now() + 2000);

client.getOrder(
  { orderId: "ord_a3f9c2" },
  meta,
  { deadline },
  (err, order) => {
    if (err) {
      if (err.code === grpc.status.NOT_FOUND) return console.log("no such order");
      if (err.code === grpc.status.DEADLINE_EXCEEDED) return console.log("timed out");
      throw err;
    }
    console.log(order.id, order.status);
  },
);

Streaming modes#

gRPC’s four modes, with the natural fit for each:

Streams use the same HTTP/2 connection — no separate WebSocket handshake, no protocol upgrade. Back-pressure is built in via HTTP/2 flow control.

Deadlines, metadata, and the status taxonomy#

Three things gRPC does that REST does informally:

• Deadlines are end-to-end, not per-hop. A client sets a deadline; the server propagates it to its own downstream calls. If a deadline passes anywhere in the chain, the call is cancelled top to bottom. (REST has read/write timeouts but no native end-to-end deadline propagation.)
• Metadata is gRPC’s headers. Both directions, both initial and trailing. The convention is lowercase keys; authorization carries the bearer token; x-trace-id carries the trace ID.
• Status codes are gRPC’s own, not HTTP’s. The 16-code taxonomy:

The mental model: gRPC’s codes are more precise than HTTP’s. INVALID_ARGUMENT vs FAILED_PRECONDITION is a meaningful distinction — bad client input vs server-state conflict — that HTTP collapses into “4xx”.

Browser support — grpc-web and Connect#

The honest part: browsers don’t speak gRPC natively. They can’t because the JavaScript fetch / XHR APIs don’t expose HTTP/2 frames the way gRPC needs.

Two paths exist:

• grpc-web — a wire-compatible subset of gRPC that browsers can speak. An Envoy proxy (or any grpc-web-aware gateway) translates between grpc-web and full gRPC. Streaming is reduced to server-stream only.
• Connect (buf.build/connect) — a newer alternative that speaks three protocols on one server: gRPC, grpc-web, and a Connect-specific HTTP/JSON protocol. Same .proto, same codegen, runs on Cloudflare Workers and Vercel Edge without proxy gymnastics.

Many teams who picked gRPC for the backend keep REST or GraphQL at the edge for browsers. The gateway translates. That’s not a bug; it’s the deployment shape that wins in practice.

Variants#

Trade-offs#

What gRPC gives you:

• Codegen across 10+ languages. The polyglot story is the killer feature.
• Binary efficiency. Protobuf is 3-10x smaller than JSON; HTTP/2 multiplexes; both add up to material throughput wins on internal traffic.
• Streaming as a first-class concept. Four streaming modes, all over the same transport, all with native back-pressure.
• Deadlines and cancellation that propagate. End-to-end, not per-hop.
• A schema that doubles as documentation. The .proto file is the canonical contract.
• Tight error semantics. The 16-code taxonomy gives clients precise discrimination of failure modes.

What gRPC costs you:

• Browser support is awkward. grpc-web or Connect, plus a proxy if you go grpc-web.
• Curl-debuggability is gone. grpcurl works but isn’t curl. Browser dev-tools can’t introspect binary Protobuf without a plugin.
• Protobuf has its own learning curve. Field numbering, reserved, oneof, evolution rules — there are footguns.
• Operational tooling lags HTTP/JSON. Many proxies, gateways, and observability tools have weaker gRPC support than HTTP.
• Public-API onboarding is heavier. Integrators have to run protoc, manage a stub, depend on Protobuf libraries. Worth it for high-volume integrators; too much for low-volume ones.

Common pitfalls#

• Renumbering Protobuf fields. Wire-incompatible; breaks every deployed client. Always reserved removed numbers.
• Not setting deadlines. A gRPC call without a deadline can hang forever, holding a stream slot. Always set one.
• Conflating HTTP status with gRPC status. A gRPC server doesn’t return HTTP 4xx/5xx for application errors — it returns OK HTTP with a gRPC trailer status. Logging tools that look at HTTP status will miss gRPC errors entirely.
• Using proto3 without optional for fields that might be unset. Zero and absent are the same. If you need to distinguish, mark the field optional (available since proto3.15).
• Ignoring UNAVAILABLE retry semantics. UNAVAILABLE is the retry-with-backoff signal. INTERNAL is not. Treating them the same is a common mistake.
• One giant .proto file. Split by service. Use Protobuf packages.
• gRPC for a public API without considering the integrator cost. Most external consumers will need a REST or GraphQL surface alongside.

Related building blocks#

• REST — The Architectural Style — the alternative for resource-oriented APIs.
• RESTful API Design in Practice — REST’s practical playbook.
• GraphQL — A Query Language for APIs — the typed-schema alternative with client-shaped responses.
• REST vs GraphQL vs gRPC — Comparison — REST vs GraphQL vs gRPC, head-to-head.
• Remote Procedure Calls (RPCs) — the RPC pattern gRPC implements.
• Binary Data Formats — Protobuf, MessagePack, Avro — Protobuf, FlatBuffers, MessagePack in depth.
• The Evolution of HTTP — 1.1, 2, 3 — the HTTP/2 transport gRPC stands on.