Get Started with Mistral AI OCR: Plug-and-Play Wrapper with Visual Outputs in 6 Steps

Skip setup headaches, this 6-step guide shows how to use my minimal Mistral OCR wrapper that extracts text and embeds image snippets directly in the results.

If you’ve got a use case where you want tables, charts, diagrams, or other visual elements included in the OCR output, then right now Mistral AI is one of the best options out there.

Source Image & OCR Output

That said, even though it’s pretty accurate most of the time, it does occasionally miss a few image clippings.
For example, in one case with two labelled plant diagrams, it extracted both the labels perfectly but only picked up one of the images.
It’d also be nice if we could include a custom prompt along with the OCR request, like you can with some other AI models. But overall, it still performs great for what it does.

Anyway, if your use case sounds like that, I built a minimal Python wrapper to make it easy.

Using the Wrapper

All you need to do is import the Mistral client:

from mistral_ai import mistral_client

and then call either of these methods:

# OCR an image
await mistral_client.ocr_image("path/to/image.png")

# OCR a PDF
await mistral_client.ocr_pdf("path/to/document.pdf")

In response, you’ll get a Markdown string with the extracted text and embedded images encoded in Base64.

Get Started in 6 Quick Steps

1.Clone the repository from GitHub.

git clone https://github.com/gamesbyBAE/mistral-ocr-with-embedded-images

2. Install the 2 required packages:

• mistralai: The core Mistral library
• python-dotenv: To read values from a file & set them as environment values.

Install the packages by running:

pip install -r requirements.txt

⚠️ Highly recommended to create a venv before installing the packages.

3.Generate an API key in Mistral’s dashboard.

API Keys section in Mistral Dashboard

4. Create a .env file at the root of the directory.

.env File at the Project Root

5.In this newly created .env file, add the following:

# Put your Mistral API key here
MISTRAL_API_KEY=YOUR_API_KEY_HERE

# Optional: override model name
MISTRAL_OCR_MODEL=mistral-ocr-latest

6.Check out main.py, which demonstrates how to use the Mistral wrapper.
It performs OCR on the included sample image and PDF file.

Upon its successful execution, you’ll see the outputs stored as Markdown files:

• /output/image_ocr.md
• /output/pdf_ocr.md

And we are done! 🎉
This wrapper exists to simplify the process of running OCR on images or PDFs with visual elements embedded in the output, hopefully saving you some dev time.

If in need, feel free to reach out to me @MayankBagchi.dev
See ya!