Inference API

The Inference API enables you to run AI models on-device. mimOE supports two types of AI inference:

Generative AI

Generate text, have conversations, and create embeddings using LLM models (GGUF format).

Use cases:

• Chat applications and conversational AI
• Text generation and completion
• Semantic search with embeddings
• Code assistance

Generative AI Guide: OpenAI-compatible API for chat completions and embeddings

Predictive AI

Run classification, regression, and other predictive models for real-time inference.

Use cases:

• Image classification
• Anomaly detection
• Sensor data analysis
• Real-time predictions

Coming Soon

Predictive AI inference documentation is coming in a future release.

Choosing the Right Approach

Need	Use
Chat, text generation, Q&A	Generative AI
Semantic search, similarity	Generative AI (embeddings)
Classification, regression	Predictive AI (coming soon)
Sensor/time-series analysis	Predictive AI (coming soon)