Key points
- Local wins on privacy, offline use and no per-token fees; it loses on memory limits and speed with big models.
- As a rule of thumb, a 4-bit model needs about 0.6 GB of VRAM per billion parameters, plus room for context.
- A hosted endpoint gives a 100,000-token window and 8 concurrent requests with no hardware.
- Drafting an 80,000-word novel in scene-sized calls costs about $0.12 per pass at $0.15 and $0.50 per 1M tokens.
What story writing demands from a model
Fiction is harder on a language model than chat. It needs a consistent voice over many thousands of words, memory for names, places and promises made in chapter two, and a willingness to write dark or mature material when the story calls for it. Many assistant-tuned models refuse or soften those scenes, which is why writers look for uncensored options.
Four properties decide how good a setup feels:
- Size and quality of the model. Larger models hold characters and subtext better.
- Context length. It limits how much of the story the model sees at once.
- Speed. Slow generation breaks the flow of working with a draft.
- Content policy. A model that refuses a scene mid-chapter wastes your time.
Local setups and hosted APIs trade these differently, so the right choice depends on your hardware and how you work.
Local setups: what you can run
Popular ways to run models on your own machine include Ollama, LM Studio, KoboldCpp and llama.cpp. All of them load quantised weights and expose a chat interface or a local API. The page on free uncensored LLM options covers the no-cost route in more depth.
Memory is the limit. A rough rule: a 4-bit quantised model uses about 0.6 GB of VRAM per billion parameters, plus the cache that grows with context.
| Model size | Approx. weights at 4-bit | Typical hardware |
|---|---|---|
| 7B to 8B | about 5 GB | 8 GB graphics card or a recent laptop |
| 12B to 14B | about 8 GB | 12 to 16 GB graphics card |
| 30B to 35B | about 20 GB | 24 GB graphics card |
| 70B | about 40 GB | Two large cards or a Mac with lots of unified memory |
These figures are estimates and change with quantisation and context. Check the model card of anything you download. Prose quality tends to improve with size, so a 7B model on a laptop is fine for brainstorming and rougher for sustained scenes.
The context problem
The cache that stores attention state grows with every token. On a 24 GB card running a mid-size model, a long context can eat several gigabytes, so people shorten the window to 8,000 or 16,000 tokens. That equals roughly 6,000 to 12,000 words. A novel is ten times that, so you need a memory strategy whatever you run: a story bible, a rolling summary and the last scene pasted in.
A hosted endpoint changes this limit. Here the window is 100,000 tokens, shared by prompt and reply, which is on the order of 75,000 words in total. You still should not paste an entire manuscript on every call, since input is billed per token, but you can include a full bible, a long summary and several chapters of reference without trimming hard.
Hosted API for long fiction
A hosted OpenAI-compatible API turns the model into a URL. You get the same prompts and tools as local, no VRAM planning, and several requests at once: 8 concurrent on a full key, which lets you draft three variants of a scene in parallel. Details are in the docs.
What you give up: weights you control, offline use, and the zero marginal cost of a model on hardware you already own. Prompts are not used for training, but your text does leave your machine, so keep local anything that must not. The model is one open-weight model tuned to answer without content refusals for lawful adult use, with a hard rule against sexual content involving minors. If you are comparing providers, see what to look for in an uncensored LLM provider.
A scene-by-scene drafting loop
This loop works the same locally or hosted. It keeps a story bible in the system message, a running summary, and the end of the previous scene, then writes one scene at a time. Change the base URL to your local server if you run it offline.
from openai import OpenAI
client = OpenAI(base_url="https://api.bestuncensoredllm.com/v1", api_key="YOUR_KEY")
bible = open("story_bible.txt").read() # characters, setting, tone, rules
outline = open("outline.txt").read().split("\n---\n")
recap = ""
chapters = []
for i, beat in enumerate(outline, start=1):
tail = chapters[-1][-3000:] if chapters else ""
resp = client.chat.completions.create(
model="uncensored",
temperature=0.9,
top_p=0.95,
presence_penalty=0.3,
messages=[
{"role": "system", "content": bible},
{"role": "user", "content":
f"Story so far (summary):\n{recap}\n\nEnd of previous scene:\n{tail}\n\n"
f"Write scene {i}, about 1500 words. Beat: {beat}"},
],
)
scene = resp.choices[0].message.content
chapters.append(scene)
summ = client.chat.completions.create(
model="uncensored",
temperature=0.3,
messages=[{"role": "user", "content": f"Update this running summary in under 300 words.\n\nSummary:\n{recap}\n\nNew scene:\n{scene}"}],
)
recap = summ.choices[0].message.content
open("draft.txt", "w").write("\n\n".join(chapters))Two details matter. The summary step uses a lower temperature so facts stay stable. And the "end of previous scene" tail gives the model the voice to continue. Edit scenes by hand before the next iteration, since errors you leave in become canon for the next call.
Cost: hardware versus tokens
Take an 80,000-word novel, which is about 107,000 tokens of text (100 tokens is roughly 75 words). Draft it in 54 scenes of about 1,500 words, around 2,000 tokens each. Each call sends about 6,000 input tokens: bible, summary, tail and instructions.
Output: 54 x 2,000 = 108,000 tokens, which is 0.108 x $0.50 = $0.054. Input: 54 x 6,000 = 324,000 tokens, which is 0.324 x $0.15 = $0.0486. Add the summary calls, about 54 x 1,200 input and 300 output tokens: $0.0097 + $0.0081. The total for a full pass is roughly $0.12.
Even five complete rewrites cost about $0.60. Compare that with the price of a graphics card with 24 GB of memory, then add electricity. For most writers who draft a few books a year, tokens are cheaper than hardware. If you already own the card and write daily, local costs only power. Check other providers' current pricing for your own comparison.
Credit is prepaid, never expires and failed or refused requests are not charged. Top-ups from $20 to $500 are paid by crypto. See pricing.
How to choose
- Pick local if you own a 16 to 24 GB card, write offline, or handle text that cannot leave your machine, and you enjoy tuning.
- Pick hosted if you have a laptop, want a long window, work from several devices, or run batch jobs such as generating many variants.
- Use both if you outline locally and send the long, heavy drafting passes to the API.
Test before committing. The API key page offers a one-click $0.50 trial credit for new users, enough to draft dozens of scenes and judge the prose against your local model.
