Python Sockets — TCP Client/Server

What it is#

The Python socket module exposes the BSD TCP socket API: socket(), connect(), listen(), accept(), send() / sendall(), recv(), close(). TCP needs a few more calls than UDP because TCP is connection-oriented — the server listens and accepts connections; each accepted connection becomes its own socket; the client connects once and then reads/writes a byte stream.

The two trickiest things about TCP socket code are framing (TCP is a stream, not a message — you have to mark boundaries yourself) and concurrency (one accept returns one socket; serving many clients at once means threads, processes, selectors, or asyncio). Both are textbook traps; both have well-known patterns. This writeup shows the minimal correct code and then the patterns that scale.

When to use it#

Use raw socket for TCP when:

• Learning or teaching. TCP behaviour is best understood from the wire up; raw sockets make every choice visible (framing, partial reads, half-close).
• Implementing a custom wire protocol. Database wire formats, RPC frameworks, custom binary protocols. Anything that isn’t HTTP.
• Building tools. Port scanners, simple proxies, protocol-fuzzing scripts, packet replayers.
• Tight performance work. When http.server or even aiohttp add overhead you can’t tolerate.

Reach for higher-level libraries instead when:

• You want HTTP. httpx, requests, aiohttp, urllib3. Don’t reimplement HTTP/1.1’s quirks.
• You want async at scale. asyncio.start_server and asyncio.open_connection are much nicer than selectors.
• You need TLS. ssl.wrap_socket works but is fiddly; httpx and aiohttp handle cert validation and SNI for you.

How it works#

Minimal TCP server#

import socket

HOST = '0.0.0.0'
PORT = 9000

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((HOST, PORT))
sock.listen(128)
print(f'listening on tcp/{PORT}')

while True:
    conn, addr = sock.accept()
    with conn:
        print(f'connected by {addr}')
        while True:
            data = conn.recv(4096)
            if not data:
                break              # peer half-closed; recv returned 0
            conn.sendall(b'echo: ' + data)

The cast of calls:

1. socket(AF_INET, SOCK_STREAM) — TCP socket. SOCK_STREAM is what makes it TCP.
2. SO_REUSEADDR — allows quick rebind after restart; without it, TIME_WAIT from a prior bound socket on the same port blocks the new bind for ~60 s.
3. bind + listen(backlog) — claim the port and start accepting. The backlog is the kernel’s accept-queue length.
4. accept() — block until a client completes the three-way handshake. Returns a new socket for that connection and the peer’s address. The original sock keeps listening.
5. recv(n) — read up to n bytes from the stream. Returns b'' when the peer half-closes (FIN). Less than n is normal.
6. sendall(data) — write all of data. Wraps send, which can return short writes.

Minimal TCP client#

import socket

SERVER = ('127.0.0.1', 9000)

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    sock.settimeout(5.0)
    sock.connect(SERVER)
    sock.sendall(b'hello')
    reply = sock.recv(4096)
    print(f'got {reply!r}')

connect blocks through the three-way handshake. sendall pushes bytes into the OS send buffer; the kernel handles segmentation and retransmits. recv(4096) returns whatever’s currently buffered — could be 1 byte, could be 4096, could be partial.

Framing — TCP is a byte stream#

TCP does not preserve message boundaries. Two sendall(b'hello') calls may arrive at the receiver as one recv of b'hellohello', or as two separate reads of b'hello', or as b'he' then b'llohello'. You must frame messages yourself.

Three common framing schemes:

Length-prefixed (binary, the default for most binary protocols):

import struct
import socket

def send_msg(sock, payload: bytes) -> None:
    sock.sendall(struct.pack('!I', len(payload)) + payload)

def recv_exact(sock, n: int) -> bytes:
    buf = bytearray()
    while len(buf) < n:
        chunk = sock.recv(n - len(buf))
        if not chunk:
            raise ConnectionError('peer closed mid-message')
        buf.extend(chunk)
    return bytes(buf)

def recv_msg(sock) -> bytes:
    header = recv_exact(sock, 4)
    (length,) = struct.unpack('!I', header)
    return recv_exact(sock, length)

Delimiter-based (text protocols, HTTP/1.1, SMTP, IMAP):

def recv_until(sock, delim=b'\n') -> bytes:
    buf = bytearray()
    while True:
        chunk = sock.recv(1)
        if not chunk:
            raise ConnectionError('peer closed mid-line')
        buf.extend(chunk)
        if buf.endswith(delim):
            return bytes(buf)

(Real text protocols read in larger chunks and keep a buffer between calls — single-byte reads are slow.)

Fixed-size (rare; record-oriented binary). Every message is exactly N bytes; just call recv_exact(sock, N).

Serving many clients#

A single-threaded loop with accept then recv blocks on every connection. Three ways to serve concurrently:

Threads (one per connection):

import socket
import threading

def handle(conn, addr):
    with conn:
        while True:
            data = conn.recv(4096)
            if not data:
                return
            conn.sendall(b'echo: ' + data)

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('0.0.0.0', 9000))
sock.listen(128)

while True:
    conn, addr = sock.accept()
    threading.Thread(target=handle, args=(conn, addr), daemon=True).start()

Fine up to a few hundred concurrent connections; threads cost memory and the GIL serialises Python work.

Non-blocking + selectors (one thread, many connections):

import selectors
import socket

sel = selectors.DefaultSelector()

def accept(server_sock):
    conn, addr = server_sock.accept()
    conn.setblocking(False)
    sel.register(conn, selectors.EVENT_READ, read)

def read(conn):
    data = conn.recv(4096)
    if not data:
        sel.unregister(conn)
        conn.close()
        return
    conn.sendall(b'echo: ' + data)

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind(('0.0.0.0', 9000))
server.listen(128)
server.setblocking(False)
sel.register(server, selectors.EVENT_READ, accept)

while True:
    for key, _ in sel.select():
        key.data(key.fileobj)

selectors wraps the best polling primitive available (epoll on Linux, kqueue on macOS/BSD, select elsewhere). Scales to tens of thousands of connections.

asyncio (modern Python convention):

import asyncio

async def handle(reader, writer):
    while True:
        data = await reader.read(4096)
        if not data:
            break
        writer.write(b'echo: ' + data)
        await writer.drain()
    writer.close()
    await writer.wait_closed()

async def main():
    server = await asyncio.start_server(handle, '0.0.0.0', 9000)
    async with server:
        await server.serve_forever()

asyncio.run(main())

Almost always the right choice for new TCP server code in Python.

Close, shutdown, half-close#

TCP supports half-close: one side sends FIN (sock.shutdown(SHUT_WR)) and stops sending, but can still read. close() releases the socket entirely. A common bug is calling close() when you meant shutdown(SHUT_WR) — the peer sees RST instead of a clean FIN.

A recv returning b'' means the peer half-closed (sent FIN). It does not mean the connection is gone — you can still send.

Variants#

• socketserver module. Stdlib threading TCP server with handler classes. Fine for quick servers; verbose for anything custom.
• asyncio.start_server / open_connection. The recommended modern API. Yields (reader, writer) pairs with awaitable read, readline, readexactly, drain.
• Trio / anyio. Structured-concurrency frameworks. Cleaner cancellation semantics than asyncio for non-trivial servers.
• TLS. ssl.create_default_context() then context.wrap_socket(sock, server_hostname=...). Verifies certs and selects ciphers correctly out of the box.
• socket.create_connection((host, port), timeout). Convenience that handles DNS, IPv4/IPv6, and connect timeout in one call.
• SO_KEEPALIVE. Periodic TCP probes to detect dead peers. Off by default; defaults are slow (2 hours on Linux). Tune TCP_KEEPIDLE, TCP_KEEPINTVL, TCP_KEEPCNT for shorter detection.

Trade-offs#

Other tensions:

• Framing complexity vs flexibility. Length-prefix is fastest and unambiguous; delimiter is more debuggable (you can telnet to a line-based server). Pick by use case.
• sendall vs send. sendall loops internally to write everything; send may return early on partial write. Always use sendall unless you have a specific reason.
• recv buffer size. Larger buffers (8192, 65536) reduce syscall count but waste memory if most messages are small. 4096 is a defensible default.
• Nagle and TCP_NODELAY. Enabled by default, Nagle batches small sends. For interactive RPCs (small request, small response, latency-sensitive), set TCP_NODELAY. For bulk transfer, leave Nagle on.
• Backlog size in listen. The kernel queues accepted-but-not-accept()-ed connections here. Default 128; raise to several thousand on busy servers (and net.core.somaxconn to match).

Common pitfalls#

• Assuming one recv returns one message. TCP is a byte stream. Always frame (length-prefix, delimiter, or fixed-size).
• Forgetting SO_REUSEADDR on restart. Without it, restarting a server fails for ~60 s because TIME_WAIT from the previous bind still holds the port.
• Calling recv with too small a buffer in a loop. recv(1) works but is dramatically slower than recv(4096) and buffering yourself. Read in chunks.
• Not handling recv returning b''. Empty bytes means the peer half-closed. Treat it as end-of-stream and exit the read loop.
• Using send instead of sendall for important writes. send may write only part of your data. sendall is the safe default.
• Closing instead of shutting down. close sends RST if there’s buffered data; the peer sees an error instead of a clean FIN. Use shutdown(SHUT_WR) then drain reads then close.
• Ignoring TimeoutError from connect. A connect to an unreachable host can hang for the OS’s default TCP connect timeout (minutes). Always set settimeout on client sockets.
• Threads per connection at scale. Fine for hundreds; breaks for thousands. Switch to asyncio before you hit the wall.
• TLS without server_hostname. Without it, wrap_socket skips SNI and may skip name verification on the server cert. Always pass it on the client.
• Forgetting Nagle interaction with delayed-ACK. Small interactive sends can stall 40 ms when both are enabled. TCP_NODELAY for RPCs.

Related building blocks#

• TCP Fundamentals
• Python Sockets — UDP Client/Server
• What Is the Transport Layer?
• HTTP — Requests, Responses, Status Codes, Headers
• Tools — ping, traceroute, tcpdump, curl, dig