Baby Language Model in One Line

Try these working examples right in your terminal:

Quick Demo (no files needed)

echo "AI is awesome. AI and AWK together are even better. 
AWK is fast. AI is smart. Long live AWK in the age of AI." | 
awk '{for(i=1;i<=NF;i++) count[$i]++} END{for(w in count) print count[w], w}' | sort -nr | head -12

Better Version (case-insensitive + punctuation removed)

awk '{
    gsub(/[^a-zA-Z0-9 ]/, " "); 
    for(i=1;i<=NF;i++) {
        w=tolower($i); 
        if(w!="") count[w]++
    }
} END {
    for(w in count) print count[w], w
}' corpus.txt | sort -nr | head -20

Pro tip: Create a corpus.txt file and run the second command for real text analysis.