lua-http

daurnimator

October 2017

What is lua-http?

  • HTTP Library for Lua
  • Optionally asynchronous (including DNS lookups and TLS)
  • Supports HTTP(S) version 1.0, 1.1 and 2
  • Functionality for both client and server
  • Compatible with Lua 5.1, 5.2, 5.3 and LuaJIT

History

  • Started as experiment in writing a HTTP 2 client
  • To test the client, I needed a server...
  • Realised that lua doesn't have a good HTTP 1 client: I should support that too
  • ... and I needed to test that too
  • Scope creeped to be library for all things HTTP for Lua

Other projects

Every application lua gets embedded in implements their own new (non-feature complete) network protocol libraries

Goal

  • Be go-to http library for all Lua developers
  • Support all standardised HTTP features
  • Does not want to be everything
  • Library to build your own http server/framework
  • Even if you don't want to use lua-http's connection objects, consider using e.g. http.headers module

Compatibility

  • Works anywhere except windows
  • Due to use of cqueues, can work inside of other programs and not block their main loops

Semantics

  • Many HTTP2 semantics are used for HTTP1
    • Method/path/etc. are request headers :method, :path
    • Status code is just another response header :status
  • Use same classes for both client and server
  • All potentially blocking operations take a timeout argument
  • Return nil, err_msg, errno on failure

Avoided pitfalls

  • Multiple headers e.g. Set-Cookie
  • Timeouts apply to total operation, not parts
  • Buffering vs Streams
  • Header field formats
  • Partial failures
    • failed partial reads are :unget back into socket for retry
  • Redirects

Why cqueues?

  • Nature of HTTP2 requires a scheduler
  • HTTP1 does too for pipelining
  • Lua has no native main loop
  • cqueues is a composable main loop
    • Will work with other main loops without blocking them.

Works Sync or Async

  • When run outside of a cqueues managed coroutine, is blocking => great for quick scripts or writing utils
  • When run inside of a cqueues managed coroutine, will yield back to controller

Features

  • Good/correct SSL/TLS checking
  • Proxies
    • Will pick up from env vars
  • Follows redirects
  • Pipelining
  • Websocket library
  • Compat APIs to help migration

Client Example

local http_request = require "http.request"
local my_req = http_request.new_from_uri("http://httpbin.org/get")
local headers, stream = assert(my_req:go())
for field, value in headers:each() do
    print(field, value)
end
local body = assert(stream:get_body_as_string())
print(body)

Server Example

server_hello.lua

Maturity

  • The client module is good for use anywhere today.

  • The server module has a couple of issues left before it should be used on the wide-open internet.
    e.g. Support for MAX_CONCURRENT_STREAMS is needed to prevent resource exhaustion for a http2 server.

Future work

  • Automatic cookie store
  • Connection pooling

Installing

Questions?

Homepage: https://github.com/daurnimator/lua-http/

Docs: https://daurnimator.github.io/lua-http/