two much labs / quickwake

Lab project 011 · research · early experiments

Fast starts. Shared GPUs.

A model answers quickly if it is already loaded. Keeping every model loaded uses expensive GPU memory even when nobody is asking; unloading it makes the next person wait. Quickwake is research into how far that tradeoff can be reduced.

Where it stands Talk to us

Three ways to serve a model nobody is using right now

  1. Always warmFast first answerThe model stays loaded. You pay for the GPU memory whether or not anyone asks.
  2. Scale to zeroCheap while idleThe model is unloaded. The next request waits while it comes back.
  3. What Quickwake is afterFast first answer on shared capacityA target for the research, not a result.

A map of the tradeoff, not a ranking. We have not earned the third row yet.

/what

Keep models close, share the GPU

We are investigating a serving runtime built on four ideas, starting from existing open models and published research.

01

Weights in shared memory

Keep model weights in a shared CPU-memory tier, so an inactive model needs less dedicated GPU memory.

02

Fast paths to the GPU

Use fast CPU-to-GPU memory access so work can begin without copying a whole model across first.

03

Shared GPU scheduling

Schedule requests across shared GPU compute, balancing responsiveness, memory use and contention.

04

Measure the whole request

Time from a cold request to the first useful output, plus what it costs to keep things ready in between.

/status

Where it stands

Early research. We publish what works and what doesn’t, and experiments that failed stay on the record.

  • Research and prior art

    A review of existing mechanisms and published results, and where they stop.

    written
  • Benchmark plan

    What we measure, the fairness rules, and the evidence needed before claiming a win.

    written
  • Cold-start experiments

    Loading, compression and small-starter approaches tried on one small machine. Some helped; several did not, and we kept those results.

    target not met yet
  • Benchmark harness

    Replaying real request patterns with timing, cost and output checks for every run.

    in progress
  • Target hardware

    The first run on a system with shared CPU and GPU memory and GPU partitioning.

    not started
  • Head-to-head comparison

    Against normal serving on the same hardware, a commercial serverless GPU service, and an always-warm deployment.

    not started
What we are not claiming. We have not shown that Quickwake beats a commercial service, the research baselines or a well-tuned conventional deployment. Caching, worker revival, snapshots and GPU sharing already exist, and combining them is not automatically new. “Zero cold start” is an ambition, not a guarantee. We will publish numbers when there is a fair comparison to run them on.

Honest limits

What it doesn't do yet

  • It is a research project, not a serving product.
  • Our startup target has not been reached.
  • Switching between models already in memory is not the same as starting on a fresh machine; we measure both, and the second is the hard one.
  • Nothing has been tested yet on the hardware the approach is designed for.

/next

What comes next

  1. Finish the benchmark harness, so a conventional baseline can be measured first.
  2. Freeze a pilot workload and success criteria before spending on hardware.
  3. Run the first experiment on the target hardware.
  4. Publish the method and the results, including what did not work.

Get involved

Serving many models on shared GPUs?

The most useful help right now is a real workload shape: how many models, how often each is called, and how long a first answer may take.

Share a workload