Eino LLM Application Development Framework

Overview

Eino (pronounced similarly to "I know") is the ultimate LLM (Large Language Model) application development framework in Golang. Inspired by frameworks like LangChain and LlamaIndex, Eino emphasizes simplicity, scalability, reliability, and effectiveness, aligning with Golang programming conventions. It provides a comprehensive set of tools and abstractions to streamline AI application development.

Key Features

• Rich Components: Encapsulates common building blocks like ChatModel, Tool, ChatTemplate, Retriever, and more.
• Powerful Orchestration: Supports graph and chain-based orchestration for complex workflows.
• Complete Stream Processing: Handles streaming data efficiently across components.
• Highly Extensible Aspects (Callbacks): Allows custom logging, tracing, and metrics.
• Type Safety and Concurrency Management: Ensures robust and safe application development.

Quick Walkthrough

Using a Component Directly

model, _ := openai.NewChatModel(ctx, config) // create an invokable LLM instance
message, _ := model.Generate(ctx, []*Message{
    SystemMessage("you are a helpful assistant."),
    UserMessage("what does the future AI App look like?")})

Creating a Simple Chain

chain, _ := NewChain[map[string]any, *Message]().
           AppendChatTemplate(prompt).
           AppendChatModel(model).
           Compile(ctx)

chain.Invoke(ctx, map[string]any{"query": "what's your name?"})

Creating a Graph with Tool Calls

graph := NewGraph[map[string]any, *schema.Message]()

_ = graph.AddChatTemplateNode("node_template", chatTpl)
_ = graph.AddChatModelNode("node_model", chatModel)
_ = graph.AddToolsNode("node_tools", toolsNode)
_ = graph.AddLambdaNode("node_converter", takeOne)

_ = graph.AddEdge(START, "node_template")
_ = graph.AddEdge("node_template", "node_model")
_ = graph.AddBranch("node_model", branch)
_ = graph.AddEdge("node_tools", "node_converter")
_ = graph.AddEdge("node_converter", END)

compiledGraph, err := graph.Compile(ctx)
if err != nil {
    return err
}
out, err := r.Invoke(ctx, map[string]any{"query":"Beijing's weather this weekend"})

ReAct Agent Implementation

Eino provides a complete implementation of the ReAct Agent, which uses graph orchestration for type checking, stream processing, and concurrency management.

handler := NewHandlerBuilder().
  OnStartFn(
    func(ctx context.Context, info *RunInfo, input CallbackInput) context.Context {
        log.Infof("onStart, runInfo: %v, input: %v", info, input)
    }).
  OnEndFn(
    func(ctx context.Context, info *RunInfo, output CallbackOutput) context.Context {
        log.Infof("onEnd, runInfo: %v, out: %v", info, output)
    }).
  Build()

compiledGraph.Invoke(ctx, input, WithCallbacks(handler))

Framework Structure

Eino is divided into several parts:
- Eino: Core framework with type definitions, streaming mechanisms, and orchestration capabilities.
- EinoExt: Component implementations, callback handlers, and tools like evaluators and prompt optimizers.
- Eino Devops: Visualized development and debugging tools.
- EinoExamples: Example applications and best practices.

Documentation

For detailed documentation, visit the Eino User Manual. For a quick start, check out Eino: Quick Start.

Dependencies

• Go 1.18 and above.
• kin-openapi for OpenAPI JSONSchema implementation.

Security

To report security issues, contact Bytedance Security via their security center or vulnerability reporting email.

Contact Us

• Issues: GitHub Issues
• Lark Group: Scan the QR code below to join our user group.

License

This project is licensed under the Apache-2.0 License.

About

Eino is the ultimate LLM/AI application development framework in Golang. Learn more at www.cloudwego.io/zh/docs/eino/.

Topics

• AI
• AI Framework
• LangChain
• LLM Application