TimeCapsuleLLM

Language models trained only on texts from a single time and place: so far, London and the English-speaking world of 1800–1875.

About

Most language models learn from the modern internet, so even when asked to imitate the past they think in the present. TimeCapsuleLLM models are trained on books, newspapers, letters and legal documents printed between 1800 and 1875, so they write with the vocabulary, assumptions and blind spots of that period. Models from v2 onward are trained from scratch on period text alone; v1 was built on Microsoft's Phi-1.5.

The project was started independently by Hayk Grigorian and is now carried out under the academic supervision of Dr. Hamed Yaghoobian, with an affiliated research collaboration at Muhlenberg College and Georgia State University.

Try it

The demo runs on Hugging Face and includes v2 (London, 1.2B) and v3 mini (English, 500M). They are base models: give one the beginning of a letter, a news report or a story and it continues the text. They are trained to continue text, rather than follow instructions or reliably answer questions.

Open the demo on Hugging Face

The models reproduce the writing of their period, including its racism, sexism and colonial attitudes. Their output is generated text, often inaccurate, and should not be read as historical fact.

Paper

TimeCapsule: Generative Hallucination as a Method for Historical Sensemaking
Hayk Grigorian and Hamed Yaghoobian. First-author, peer-reviewed paper presented at ACM Creativity & Cognition 2026, published in the Proceedings of the 2026 Conference on Creativity and Cognition.

The paper presents a 1.2-billion-parameter model trained only on Victorian texts as a generative archive isolated from modern language. It reports 45.4% lower perplexity than a GPT-2 baseline on held-out Victorian prose, and a study in which two humanities scholars mistook about 40% of genuine Victorian excerpts for machine-generated text. Its training data excludes texts published after 1875, and the paper argues that this makes its hallucinations useful as probes into nineteenth-century ways of thinking.

Models

All releases are on Hugging Face, in the TimeCapsuleLLM collection.

Public model releases
ModelTraining textNotes
v3 mini (eval 1), 500MEnglish print, Britain and the US, 1800–1875Newest. Llama architecture, trained from scratch.
v2, 1.2BLondon, 1800–1875Largest London model. Llama architecture, trained from scratch.
v2 mini (eval 2), 200MLondon, 1800–1875 (sample)Evaluation run to check the v2 tokenizer.
v2 mini (eval 1), 300MLondon, 1800–1875 (sample)Evaluation run; known tokenizer spacing issue.
v1London, 1800–1875Built on Microsoft's Phi-1.5.

Training costs

TimeCapsuleLLM is self-funded. Each model was trained on a single rented GPU; costs are as reported on the model cards.

Reported training costs
ModelGPUTraining timeCost
v3 mini, 500M1 × H100 SXM (RunPod)about 21 hours$85
v2, 1.2B1 × H100 SXM (RunPod)117 hours 51 minutes$340.97
v2 mini (eval 2), 200M1 × A100 SXM (RunPod)5 hours 52 minutes (7 hours 28 minutes billed at $1.49/hour)$11.12
v2 mini (eval 1), 300MA100 SXM (rented)not publishednot published
v1A100 SXM (rented)not publishednot published
v0 and v0.5GeForce RTX 4060 (local)not publishedno rental cost

The three published figures total $437.09.

Press and discussion

Selected coverage

Discussion

More coverage

Cited in research and lists

Code and data

Cite

The paper:

@inproceedings{grigorian2026timecapsule,
  title         = {TimeCapsule: Generative Hallucination as a Method
                   for Historical Sensemaking},
  author        = {Grigorian, Hayk and Yaghoobian, Hamed},
  booktitle     = {Proceedings of the 2026 Conference on Creativity
                   and Cognition},
  publisher     = {Association for Computing Machinery},
  year          = {2026},
  doi           = {10.1145/3803784.3807554},
  eprint        = {2607.24750},
  archivePrefix = {arXiv}
}

The dataset:

@misc{london_llm_1800,
  author       = {Grigorian, Hayk and Yaghoobian, Hamed},
  title        = {Historic London English (1800–1875)},
  year         = {2025},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/postgrammar/london-llm-1800}}
}

Contact

Questions, collaborations or press enquiries are welcome. Send a message below, email haykgrigorian07@gmail.com, or connect on LinkedIn.




Messages are delivered by Netlify Forms and are only used to reply to you.