ODROID-HC1 LLM Performance & RAG Guide

1. Benchmarks: ODROID-HC1 (Qwen 0.5B Q4_K_M)

Testing executed on the Exynos 5422-powered ODROID-HC1 using llama.cpp build 925e11799.

Execution Mode	Thread Config	Prompt Processing (pp512)	Text Generation (tg128)
Unbound CPU	`-t 4`	4.48 t/s	2.42 t/s
Pinned to Cortex-A15 OPTIMAL	`taskset -c 4-7 -t 4`	4.49 t/s	2.55 t/s (+5.4%)

Why 4 Cores Beat 8 Cores on the HC1

The Exynos 5422 features a big.LITTLE architecture (4× Cortex-A7 + 4× Cortex-A15). Threading matrix multiplication across all 8 cores forces the 4 high-performance A15 cores to wait on the slower A7 cores at layer synchronization boundaries. Pinning execution strictly to cores 4–7 combined with the HC1's massive aluminum frame thermal dissipation yields maximum tokens-per-second.

2. Production Execution Commands

Optimal Live Chat / Completion

taskset -c 4-7 ./build/bin/llama-completion -m qwen0.5b.gguf -p "The top 3 reasons to use a single-board computer are:" -t 4

Live Web Pipeline (Scrape & Grounded Summary)

taskset -c 4-7 ./build/bin/llama-completion -m qwen0.5b.gguf \
  -p "Summarize what this website is about in 2 sentences based ONLY on this text:\n\n$(curl -sL https://webcentive.com | sed 's/<[^>]*>/ /g' | tr -s ' ' | head -n 30)" \
  -t 4 -no-cnv -n 128

3. Handling Large Books & PDF Collections (RAG)

If your documents exceed what fits in memory (or are in PDF/Word format), you cannot feed them raw into llama-completion. Instead, use Retrieval-Augmented Generation (RAG) to query document collections directly from your HC1's local storage:

Option A: Python Pipeline

Install lightweight libraries to chunk PDF/Word files and retrieve relevant snippets:

pip install llama-cpp-python langchain pypdf

Step 1: Convert PDFs or Word files to raw text and slice into ~500-token chunks.
Step 2: Run a lightweight semantic search to select only the top matching chunk for your prompt.
Step 3: Feed only that target chunk to qwen0.5b.gguf to keep memory consumption minimal.

Option B: Native `llama-retrieval` Tool

llama.cpp includes a built-in vector utility for search:

Vector Embeddings: The built-in llama-retrieval binary converts raw text files into vector embeddings.
Fast Indexing: Queries run locally against pre-calculated index files stored on your HC1 drive without requiring full document parsing during inference.