Prepare, fit, evaluate, deploy

Prepare your dataset

Kethra expects training data in the JSONL chat format: one JSON object per line, each with a messages array of role/content pairs. This is the same format used by OpenAI fine-tuning, so existing datasets transfer directly.

Save your dataset as data.jsonl. A minimal example with four rows, written in the voice of a British editorial press:

{"messages":[{"role":"system","content":"You are the editorial voice of Archmount Press. Write with precision and without embellishment."},{"role":"user","content":"Introduce the autumn catalogue."},{"role":"assistant","content":"The autumn catalogue arrives with eleven new titles across history, philosophy and natural science. Each is chosen for enduring relevance, not seasonal appeal."}]}
{"messages":[{"role":"system","content":"You are the editorial voice of Archmount Press. Write with precision and without embellishment."},{"role":"user","content":"Write a rejection note to an author."},{"role":"assistant","content":"Thank you for submitting the manuscript. After careful consideration we will not be taking it forward. We read it with attention and wish you well placing it elsewhere."}]}
{"messages":[{"role":"system","content":"You are the editorial voice of Archmount Press. Write with precision and without embellishment."},{"role":"user","content":"Draft a short blurb for a book on the history of salt."},{"role":"assistant","content":"Salt shaped empires, financed wars and preserved the dead. This history follows the mineral from the evaporation pans of ancient China to the tax revolts of colonial India, tracing power through what people ate and what they paid."}]}
{"messages":[{"role":"system","content":"You are the editorial voice of Archmount Press. Write with precision and without embellishment."},{"role":"user","content":"Write an out-of-office reply for the editorial team."},{"role":"assistant","content":"The editorial office is closed from 23 December to 3 January. Submissions received during this period will be read from the fourth. We thank you for your patience."}]}

For a meaningful fit, aim for at least 50 examples. 200 or more will produce a noticeably stronger result. Each example should reflect the voice and standard you want the model to adopt consistently.

Write a voice standard

The standard is a Markdown file that describes the rules your model must follow. It is used both during fitting and to generate the eval suite that scores each version. Keep it precise and testable: vague rules produce vague evals.

Save it as voice.md:

# Archmount Press voice standard

## Tone
Write in the editorial voice of a serious British press: measured, precise,
and never condescending. Avoid hollow intensifiers and filler phrases.

## Grammar
- Use the active voice wherever the passive adds nothing.
- Do not use contractions in formal communications.
- Avoid the word "very". Use a stronger word or remove it.

## Length
Match the length to the task. A rejection note should not exceed four sentences.
A catalogue introduction should not exceed three paragraphs.

## Banned phrases
"innovative", "exciting", "passionate", "thrilled", "pleased to announce",
"unique opportunity", "in today's world".

Run kethra fit

With the dataset and standard ready, run the fit. The --base auto flag lets Kethra select the best-suited open-weight base for your task and data volume:

kethra fit \
  --data ./data.jsonl \
  --standard ./voice.md \
  --base auto

The CLI streams training progress. When the fit completes, it prints a job ID and the name of the resulting model:

Fitting     base=qwen2.5-7b  examples=204  epochs=3
Epoch 1/3   loss=1.412
Epoch 2/3   loss=0.881
Epoch 3/3   loss=0.604
Fit complete  job=fit_01j9kx3p  model=house-v1

The job ID identifies this particular run. The model name, house-v1, is derived from the first fit in your workspace. Subsequent fits produce house-v2, house-v3, and so on. You can also set a name explicitly with the --name flag.

Build and run an eval suite

An eval suite is a JSONL file of test cases. Each case has a prompt the model will receive and a criteria array describing what a passing response looks like. The criteria map directly to the rules in your standard.

Save the suite as bar.jsonl:

{"prompt":"Write a short announcement that the spring catalogue is available.","criteria":["Active voice","No contractions","No banned phrases","Under 60 words"]}
{"prompt":"Draft a note declining a speaking invitation.","criteria":["Active voice","Polite but direct","Under 50 words","No hollow intensifiers"]}
{"prompt":"Write a one-sentence description of a new book on cartography.","criteria":["Precise and specific","No filler phrases","One sentence only"]}
{"prompt":"Write a renewal reminder for a journal subscription.","criteria":["Active voice","No contractions","Polite tone","Under 40 words"]}

Run the eval against the fit:

kethra eval --suite ./bar.jsonl --job fit_01j9kx3p

The scoreboard compares the base model to house-v1:

┌──────────────────┬───────┬──────────┬───────┐
│ Metric           │ Base  │ house-v1 │ Delta │
├──────────────────┼───────┼──────────┼───────┤
│ Voice            │  0.38 │     0.72 │  +34  │
│ Standard adhere. │  0.35 │     0.74 │  +39  │
│ Accuracy         │  0.81 │     0.83 │   +2  │
├──────────────────┼───────┼──────────┼───────┤
│ Overall pass     │ 29/50 │    45/50 │  +16  │
└──────────────────┴───────┴──────────┴───────┘

Failed cases (5):
  #12  Missing: Active voice
  #23  Missing: No contractions
  #31  Missing: Under 40 words
  #38  Missing: No hollow intensifiers
  #44  Missing: Precise and specific

The fit cleared 45 of 50 cases. Inspect the five failures with kethra eval --inspect --job fit_01j9kx3p to see the full prompt, the model response, and which criteria were not met. Add examples covering those failures to data.jsonl, tighten the relevant rules in voice.md, and re-run the fit to iterate.

Deploy the model

When the eval is satisfactory, deploy to a private, versioned endpoint. The --name flag sets the model name you will use when calling the endpoint:

kethra deploy --private --name house-v1

Deployed  name=house-v1  endpoint=https://api.basisresearch.tech/v1

Your endpoint is private by default. Only callers with a valid API key for your workspace can reach it. You can issue and revoke keys from the atelier.

Call the endpoint

Set the API key in your environment, then call house-v1 in any of the three ways below.

Via the kethra CLI

export KETHRA_API_KEY=kt_live_...

kethra chat \
  --model house-v1 \
  --message "Draft a blurb for a book on maps."

Via the TypeScript client

The endpoint is OpenAI-compatible. Use the official openai package, point baseURL at Kethra, and set model to your deployed name. No other changes to your existing code are required:

import OpenAI from "openai";

const kethra = new OpenAI({
  baseURL: "https://api.basisresearch.tech/v1",
  apiKey: process.env.KETHRA_API_KEY,
});

const res = await kethra.chat.completions.create({
  model: "house-v1",
  messages: [
    {
      role: "system",
      content:
        "You are the editorial voice of Archmount Press. Write with precision and without embellishment.",
    },
    {
      role: "user",
      content: "Draft a blurb for a book on maps.",
    },
  ],
});

console.log(res.choices[0].message.content);

Via curl

curl https://api.basisresearch.tech/v1/chat/completions \
  -H "Authorization: Bearer $KETHRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "house-v1",
    "messages": [
      {
        "role": "system",
        "content": "You are the editorial voice of Archmount Press. Write with precision and without embellishment."
      },
      {
        "role": "user",
        "content": "Draft a blurb for a book on maps."
      }
    ]
  }'

Next steps

• Evals: learn how to write stronger eval suites, add regression cases, and track the delta across successive versions of your model.
• Deployment: configure versioned endpoints, set up rollback, and export your weights to your own infrastructure at any time.
• Quickstart: a faster path if you want to run a sample fit in the browser before committing a full dataset.