In August, Hum open sourced our new Lodestone large language model. With over 1,000 downloads a month 🚀 - we’re already making waves for organizations interested in training models to ingest more information at a time.
We built Lodestone to take a massive leap forward in understanding scholarly content. Most AI models today can only process inputs of 512 tokens – enough to comprehend a title, abstract, and small portion of introduction. But Lodestone can handle sequences of 4096 tokens. It’s capable of understanding a complete, mid-length research paper.
Unlike generative models (GPT-3, GPT-4, etc.), Lodestone isn’t focused on generating text; it’s focused on understanding context.
It reads papers and transforms their information into embeddings that Alchemist, Hum’s AI engine, can understand and use to build connections across the entire ecosystem of a publisher’s content.
Already on benchmark leaderboards, Lodestone is one of the highest performing models of its size and sequence length.
The Future of Lodestone
We’re busy building the Lodestone-v2-base model, which we’re aiming to implement into Alchemist by year's end. Early next year, we’ll use the v2 model as the foundation for fine-tuning a scholarly version (Lodestone-v2s). We’re aiming to build the best LLM available to scholarly publishers by deepening its academic knowledgebase, and training on scholarly-specific tasks.
Lodestone-v2 and Lodestone-v2s will form the core of Alchemist, our AI engine. When you understand content deeply, you can transfer that understanding to people. Which is why we’re aiming to understand scholarly content better than anyone on the planet.
As the model continues to be trained on volumes of science and humanities resources, Alchemist will be able to go beyond understanding what each individual piece of content is about and develop a fine-grained understanding of what individual sections, methods, or chapters within those pieces of content are about and then transfer that understanding to the people reading it.
This unlocks a world of potential applications: improved search (both content and people), summarization (including different summaries for different audiences), enhanced content recommendations, AI research assistants, and more. All in the service of understanding and engaging audiences better.
If you’re experimenting or building using AI and don’t want to go it alone, consider working with Hum. In the meantime, if you’re looking to learn more about Alchemist, our AI suite, you can read more here. If you want to try out Lodestone, check out the model page on HuggingFace.