As we navigate the ever-evolving landscape of artificial intelligence (AI), it's becoming increasingly clear that we're entering what Gartner's Hype Cycle calls "the trough of disillusionment." This phase, while it may sound discouraging, is actually a crucial step in the maturation of any transformative technology. 

At Silverchair's Platform Strategies conference last week, Dylan DiGioia, Hum’s Director of Engineering, joined experts Katherine Eisenberg (EBSCO); Emilie Delquie and Jeremy Little (Silverchair); and Dr. Mohamed Elshenawy (Sinai.ai) to share insights about AI product development & how publishers should approach AI experimentation in this context.

Understanding the Hype Cycle

Before delving into the insights shared at the conference, it's important to understand the context of the Hype Cycle. This model, developed by Gartner, describes the typical progression of emerging technologies:

  1. The Technology Trigger: A potential breakthrough sparks significant interest.
  2. The Peak of Inflated Expectations: Early publicity leads to over-enthusiasm and unrealistic expectations.
  3. The Trough of Disillusionment: Interest wanes as implementations fail to deliver on inflated promises.
  4. The Slope of Enlightenment: More instances of how the technology can benefit enterprises start to crystallize and become more widely understood.
  5. The Plateau of Productivity: Mainstream adoption starts to take off.

AI's Current Position: The Trough of Disillusionment

Nearly two years after ChatGPT's launch, which dramatically shifted the technology landscape, we find ourselves sliding into the trough of disillusionment. This stage is characterized by a more realistic view of AI's capabilities, as the initial hype gives way to more practical considerations.

As the panelists explained, the trough of disillusionment is really where progress begins. The mismatch between high expectations and reality is a necessary step in maturing products, testing use cases, and honing in on solutions to real-world problems. 

As AI tech is refined and improved, the products that aren’t adding value will fall away and more tangible and practical applications will emerge. 

Lessons from Early Adopters

The conference brought together experts from various fields to share their insights on navigating this crucial phase of AI development. Their experiences and advice offer valuable lessons for publishers striving to innovate with new AI technology.

Start Small, But Start Somewhere

Dr. Elshenawy offered a simple but powerful piece of advice: "Start small, but start." This approach allows companies to gain experience with AI technologies without committing to large-scale, potentially risky projects.

Many organizations are intimidated by the prospect of implementing AI, fearing the need for massive datasets or complex infrastructure. However, starting small allows for manageable experimentation and learning. 

Experiment with Small Datasets

DiGioia also advised publishers to "Find small datasets to prototype with. This could be 50 papers, 100 papers, 1,000 papers, or journals that you have a very friendly relationship with – so you’re able to test the impact of experiments." 

This approach allows for rapid prototyping and learning without the need for massive datasets.

Learn from Users

AI solutions, no matter how sophisticated, are only valuable if they address real user needs and pain points. 

"Agile user feedback is incredibly valuable," said Eisenberg. “Continuous user feedback helps ensure that AI development remains aligned with actual user requirements.” 

Publishers who are able to move from the ‘trough of disillusionment’ to the ‘slope of enlightenment’ are those that can quickly iterate and improve their AI implementations based on real-world usage and needs.

Publishers building their own internal AI capabilities should consider the following to solicit user feedback: 

  • Establish regular feedback loops with a diverse group of users.
  • Implement user testing sessions at various stages of AI development.
  • Create easy channels for users to report issues or suggest improvements (e.g., in-app feedback tools).
  • Use analytics to track user interactions with AI features and identify areas for improvement.

Focus on Data Access

When asked about optimizing content for AI language models, DiGioia shared an interesting insight: "Data access is more challenging, often, than data cleanliness." 

Many organizations get bogged down in data cleaning efforts, while the real bottleneck is often data accessibility. Customer data platforms (like Hum!), responsible data management practices, and other connectivity tools help ensure that relevant data can be easily accessed and used by AI systems. 

Know Your Limitations

Not every organization needs to become an AI powerhouse; sometimes, leveraging partnerships or third-party solutions is the most effective approach.

Little advised companies to be realistic about their capabilities. "Some projects are going to be more cost-effective and time-efficient if you find a partner," he said. "There is value in knowing when to tackle a project and when to outsource." 

Upskill Internally

Even when working with partners, Little and DiGioia stressed the importance of internal knowledge: "It's invaluable to upskill internally, even when you're playing with partners who deeply understand your industry." 

While partnerships are valuable, having internal AI expertise is crucial for making strategic decisions, effectively collaborating with partners, and maintaining long-term AI capabilities.

The Silver Lining of the Trough

While the trough of disillusionment might seem like a setback, it's actually a crucial phase in the development of AI technologies. As the initial hype fades, we're left with a more realistic understanding of AI's capabilities and limitations. This understanding is essential for developing practical, effective AI solutions.

The trough of disillusionment allows for:

  1. Reflection and Learning: Companies can assess what went wrong with their initial AI implementations and adjust their strategies accordingly.
  2. Focus on Specific Use Cases: Instead of expecting AI to solve every problem, developers can pinpoint exactly where it adds value.
  3. Responsible Development: With a more realistic view of AI's capabilities, there's an opportunity to develop more responsible and ethical AI systems.
  4. Improved Products: As companies refine their AI offerings based on real-world feedback, the resulting products are likely to be more useful and effective.

Looking Ahead

As we navigate through the trough of disillusionment, it's clear that AI still holds immense potential. The key is to approach its development and implementation with a balanced, realistic perspective.

By starting small, learning from users, focusing on specific use cases, and continuously improving, companies can emerge from the trough with robust, practical AI solutions.