Hum, the leading provider of AI & data intelligence solutions for publishers, today announced the open source release of their new large language model (LLM), Lodestone. 

Using Google's BERT architecture as a foundation and leveraging several improvements that have been released since then, Hum developed a novel model for processing longer text sequences. Lodestone is the highest performing model of its size and sequence length on the MTEB Leaderboard. This makes it particularly compelling for real-time applications on long text where larger models may be prohibitively expensive or slow.

"We are excited to contribute to the open source community and advance natural language processing research, while helping other organizations unlock value from content data," said Niall Little, Hum’s CTO. "Lodestone is capable of contextualizing an entire mid-length research paper, surfacing content insights that other popular models can't comprehend looking at just one or two paragraphs at a time."

Lodestone was trained on a large publicly available dataset, including over 1 million scholarly research articles and publications. The model can process text sequences of 4096 tokens to better capture topical context and nuance compared to other commonly-used LLM models. 

Key features include:

  • Long sequence embedding
  • Improved semantic understanding
  • Sentence vectorization for information retrieval, clustering, and sentence similarity tasks

Starting today, developers and enterprises can fine-tune and deploy their own models using Lodestone on Hugging Face, putting long-sequence AI applications in reach of more projects and businesses. Lodestone is also the foundational model of Alchemist, Hum's AI engine.

"This release furthers our commitment to using AI to solve challenges for publishers and societies," said Little. "We’re continuing to fine-tune the model for the needs of our publishing industry clients, but look forward to seeing how the community can build on Lodestone to advance applications for content intelligence and responsible AI."