The first time I saw my family struggle to interpret the NHS and council letters, I decided to create an application that explains these letters in plain English. Government letters are unstructured data full of dates, instructions, codes, and jargon, but mostly people only need to know three things: what it’s about, when it’s happening, and what to do next. That became the starting point for LetterLens. Purpose Purpose I aimed to reduce the anxiety people feel when dealing with government paperwork. With LetterLens, the user simply scans the letter, and the system translates it into easy-to-understand English with clear next steps. It highlights the key actions the letter expects, so users know exactly what to do. Disclaimer: LetterLens is an educational prototype, not an alternative for legal advice. Disclaimer: LetterLens is an educational prototype, not an alternative for legal advice. Disclaimer: Tech Stack (Decision-making) Tech Stack (Decision-making) Jetpack Compose: Modern declarative UI, quick prototyping, and easier state handling.
CameraX: Lifecycle-aware camera integration, seamless with compose.
Ktor: Lightweight Kotlin-native backend to classify and elaborate the letter types.
ML Kit: On-device OCR for privacy and low latency. Jetpack Compose: Modern declarative UI, quick prototyping, and easier state handling. Jetpack Compose: CameraX: Lifecycle-aware camera integration, seamless with compose. CameraX: Ktor: Lightweight Kotlin-native backend to classify and elaborate the letter types. Ktor: ML Kit: On-device OCR for privacy and low latency. ML Kit: Under the Hood(Deep Dive) Under the Hood(Deep Dive) Camerax: capture and preview Camerax: capture and preview Camerax makes capturing and analyzing images. Here's the setup to bind a preview and analyzer to the lifecycle. val cameraProviderFuture = ProcessCameraProvider.getInstance(ctx)
cameraProviderFuture.addListener({
   val provider = cameraProviderFuture.get()
   val preview = Preview.Builder()
       .setTargetAspectRatio(AspectRatio.RATIO_4_3) // keep 4:3
       .build()
       .also { it.setSurfaceProvider(view.surfaceProvider) }
   provider.unbindAll()
   provider.bindToLifecycle(
       lifecycle,
       CameraSelector.DEFAULT_BACK_CAMERA,
       preview,
       imageCapture
   )
}, ContextCompat.getMainExecutor(ctx)) val cameraProviderFuture = ProcessCameraProvider.getInstance(ctx)
cameraProviderFuture.addListener({
   val provider = cameraProviderFuture.get()
   val preview = Preview.Builder()
       .setTargetAspectRatio(AspectRatio.RATIO_4_3) // keep 4:3
       .build()
       .also { it.setSurfaceProvider(view.surfaceProvider) }
   provider.unbindAll()
   provider.bindToLifecycle(
       lifecycle,
       CameraSelector.DEFAULT_BACK_CAMERA,
       preview,
       imageCapture
   )
}, ContextCompat.getMainExecutor(ctx)) Tip: Rotate based on EXIF and crop margins; sharper, upright images improve OCR markedly. Tip: Rotate based on EXIF and crop margins; sharper, upright images improve OCR markedly. Tip: ML Kit OCR: Extract raw text on-device ML Kit OCR: Extract raw text on-device ML Kit processes the image and extracts raw text and confidence scores. val img = InputImage.fromFilePath(context, Uri.fromFile(photo))
val rec = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
rec.process(img)
   .addOnSuccessListener { onText(it.text) }
   .addOnFailureListener { e -> onError("OCR failed: ${e.message}") } val img = InputImage.fromFilePath(context, Uri.fromFile(photo))
val rec = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
rec.process(img)
   .addOnSuccessListener { onText(it.text) }
   .addOnFailureListener { e -> onError("OCR failed: ${e.message}") } Note: I keep work entirely on-device; no letter images leave the phone. Note: I keep work entirely on-device; no letter images leave the phone. Note: Ktor 'explains' endpoint: classify + extract Ktor 'explains' endpoint: classify + extract A small Ktor service classifies text and pulls deadlines/actions. routing {
   post("/explain") {
       val req = call.receive<ExplainReq>()
       val type = classify(req.text, req.hint)
       val deadline = extractDeadline(req.text, type)
       val (summary, actions, citations) = explainForType(type, req.text)
       call.respond(ExplainRes(type, deadline, summary, actions, citations))
   }
} routing {
   post("/explain") {
       val req = call.receive<ExplainReq>()
       val type = classify(req.text, req.hint)
       val deadline = extractDeadline(req.text, type)
       val (summary, actions, citations) = explainForType(type, req.text)
       call.respond(ExplainRes(type, deadline, summary, actions, citations))
   }
} Keyword heuristics (examples) Keyword heuristics (examples) NHS Appointment: “appointment”, “please attend”, “clinic”, “vaccination”, “CHI number”.


Electoral Register / Annual Canvass: “Electoral Registration Office”, “annual canvass”, “unique security code”, “register to vote”. NHS Appointment: “appointment”, “please attend”, “clinic”, “vaccination”, “CHI number”. NHS Appointment: “appointment”, “please attend”, “clinic”, “vaccination”, “CHI number”. Electoral Register / Annual Canvass: “Electoral Registration Office”, “annual canvass”, “unique security code”, “register to vote”. Electoral Register / Annual Canvass: “Electoral Registration Office”, “annual canvass”, “unique security code”, “register to vote”. Data Parsing & Classification Data Parsing & Classification Beyond the /explain endpoint, the core of LetterLens is its classifier. Government letters are messy—mixed fonts, spacing, codes, dates, so I added helpers for normalization, fuzzy matching, and deadline detection. Normalization helpers: Normalization helpers: private fun norm(s: String) = s
   .lowercase()
   .replace('’', '\'')
   .replace('–', '-')
   .replace(Regex("\\s+"), " ")
   .trim() private fun norm(s: String) = s
   .lowercase()
   .replace('’', '\'')
   .replace('–', '-')
   .replace(Regex("\\s+"), " ")
   .trim() Fuzzy matching (so “N H-S” still matches “NHS”): Fuzzy matching (so “N H-S” still matches “NHS”): private fun fuzzyRegex(token: String): Regex {
   val letters = token.lowercase().filter { it.isLetterOrDigit() }
   val pattern = letters.joinToString("\\W*")
   return Regex(pattern, RegexOption.IGNORE_CASE)
} private fun fuzzyRegex(token: String): Regex {
   val letters = token.lowercase().filter { it.isLetterOrDigit() }
   val pattern = letters.joinToString("\\W*")
   return Regex(pattern, RegexOption.IGNORE_CASE)
} Classify by domain: Classify by domain: private fun classify(textRaw: String, hint: String?): String {
   val n = norm("${hint ?: ""} $textRaw")
   if (hasAny(n, "nhs", "appointment", "vaccination")) return "NHS Appointment"
   if (hasAny(n, "electoral register", "unique security code")) return "Electoral Register"
   if (hasAny(n, "council tax", "arrears")) return "Council Tax"
   if (hasAny(n, "hmrc", "self assessment")) return "HMRC"
   if (hasAny(n, "dvla", "vehicle tax")) return "DVLA"
   if (hasAny(n, "ukvi", "visa", "biometric residence")) return "UKVI"
   return "Unknown"

} private fun classify(textRaw: String, hint: String?): String {
   val n = norm("${hint ?: ""} $textRaw")
   if (hasAny(n, "nhs", "appointment", "vaccination")) return "NHS Appointment"
   if (hasAny(n, "electoral register", "unique security code")) return "Electoral Register"
   if (hasAny(n, "council tax", "arrears")) return "Council Tax"
   if (hasAny(n, "hmrc", "self assessment")) return "HMRC"
   if (hasAny(n, "dvla", "vehicle tax")) return "DVLA"
   if (hasAny(n, "ukvi", "visa", "biometric residence")) return "UKVI"
   return "Unknown"

} Deadline extraction (supports both 12 Sept 2025 and 12/09/2025 formats): Deadline extraction (supports both 12 Sept 2025 and 12/09/2025 formats): private fun extractDeadline(raw: String, type: String? = null): String? {
   val n = norm(raw)
   return DATE_DMY_SLASH.find(n)?.value ?: DATE_DMY_TEXT.find(n)?.value
} private fun extractDeadline(raw: String, type: String? = null): String? {
   val n = norm(raw)
   return DATE_DMY_SLASH.find(n)?.value ?: DATE_DMY_TEXT.find(n)?.value
} Explain response with summary + actions + citations: Explain response with summary + actions + citations: private fun explainForType(type: String, text: String): Triple<String, List<String>, List<String>> {
   return when (type) {
       "Electoral Register" -> Triple(
           "Looks like an Electoral Register annual canvass letter.",
           listOf("Go to website", "Enter unique security code", "Confirm/update household"),
           listOf("https://www.gov.uk/register-to-vote")
       )
       "NHS Appointment" -> Triple(
           "An NHS clinic invite (likely vaccination).",
           listOf("Add to calendar", "Bring Red Book", "Call if reschedule needed"),
           listOf("https://www.nhs.uk/nhs-services/appointments-and-bookings/")
       )
       else -> Triple("Generic gov letter", listOf("Read carefully", "Follow instructions"), listOf("https://www.gov.uk"))
   }
} private fun explainForType(type: String, text: String): Triple<String, List<String>, List<String>> {
   return when (type) {
       "Electoral Register" -> Triple(
           "Looks like an Electoral Register annual canvass letter.",
           listOf("Go to website", "Enter unique security code", "Confirm/update household"),
           listOf("https://www.gov.uk/register-to-vote")
       )
       "NHS Appointment" -> Triple(
           "An NHS clinic invite (likely vaccination).",
           listOf("Add to calendar", "Bring Red Book", "Call if reschedule needed"),
           listOf("https://www.nhs.uk/nhs-services/appointments-and-bookings/")
       )
       else -> Triple("Generic gov letter", listOf("Read carefully", "Follow instructions"), listOf("https://www.gov.uk"))
   }
} Show an example API call/output: Show an example API call/output: Sample request: Sample request: POST /explain
{
   "text": "Your NHS vaccination appointment is on 25 Sept at Glasgow Clinic. Please bring your Red Book."
} POST /explain
{
   "text": "Your NHS vaccination appointment is on 25 Sept at Glasgow Clinic. Please bring your Red Book."
} Sample response: Sample response: {
   "type": "NHS Appointment",
   "deadline": "25 Sept 2025",
   "summary": "This looks like an NHS appointment invite (e.g., vaccination). When: 25 Sept. Location: Glasgow Clinic.",
   "actions": [
   "Add the appointment date/time to your calendar.",
   "Bring any requested documents (e.g., child Red Book).",
   "If you need to reschedule, call the number on the letter."
   ],
   "citations": ["https://www.nhs.uk/nhs-services/appointments-and-bookings/"]
} {
   "type": "NHS Appointment",
   "deadline": "25 Sept 2025",
   "summary": "This looks like an NHS appointment invite (e.g., vaccination). When: 25 Sept. Location: Glasgow Clinic.",
   "actions": [
   "Add the appointment date/time to your calendar.",
   "Bring any requested documents (e.g., child Red Book).",
   "If you need to reschedule, call the number on the letter."
   ],
   "citations": ["https://www.nhs.uk/nhs-services/appointments-and-bookings/"]
} Compose UI: Compose UI: A simple card shows type, deadline, summary, and actions. ElevatedCard(Modifier.fillMaxWidth()) {
   Column(
       Modifier.padding(16.dp),
       verticalArrangement = Arrangement.spacedBy(8.dp)
   ) {
       Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
           Text("Type:", fontWeight = FontWeight.SemiBold)
           Text(r.type)
       }
       Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
           Text("Deadline:", fontWeight = FontWeight.SemiBold)
           Text(r.deadline ?: "—")
       }
       Divider()
       Text("Summary", style = MaterialTheme.typography.titleMedium)
       Text(r.summary)
       Divider()
       Text("Actions", style = MaterialTheme.typography.titleMedium)
       r.actions.forEach { a -> Text("• $a") }
       Divider()
       Text("Citations", style = MaterialTheme.typography.titleMedium)
       r.citations.forEach { c -> Text(c) }
   }
} ElevatedCard(Modifier.fillMaxWidth()) {
   Column(
       Modifier.padding(16.dp),
       verticalArrangement = Arrangement.spacedBy(8.dp)
   ) {
       Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
           Text("Type:", fontWeight = FontWeight.SemiBold)
           Text(r.type)
       }
       Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
           Text("Deadline:", fontWeight = FontWeight.SemiBold)
           Text(r.deadline ?: "—")
       }
       Divider()
       Text("Summary", style = MaterialTheme.typography.titleMedium)
       Text(r.summary)
       Divider()
       Text("Actions", style = MaterialTheme.typography.titleMedium)
       r.actions.forEach { a -> Text("• $a") }
       Divider()
       Text("Citations", style = MaterialTheme.typography.titleMedium)
       r.citations.forEach { c -> Text(c) }
   }
} Results Results A letter that takes ~5 minutes to interpret can be summarized in ~10 seconds.
In my tests on NHS/council letters, the app reliably pulled dates, locations, and required items.
Clear, low-friction UX reduced the cognitive load for non-technical users. A letter that takes ~5 minutes to interpret can be summarized in ~10 seconds. In my tests on NHS/council letters, the app reliably pulled dates, locations, and required items. Clear, low-friction UX reduced the cognitive load for non-technical users. Lessons Learned Lessons Learned ML Kit OCR is surprisingly easy to set up in < 20 lines of Kotlin.
On-device AI ensures privacy (no letter leaves the phone).
Compose + CameraX makes UI binding smooth. ML Kit OCR is surprisingly easy to set up in < 20 lines of Kotlin. On-device AI ensures privacy (no letter leaves the phone). Compose + CameraX makes UI binding smooth. What's next What's next IOS version with KMP/Swift.
Multi-lingual support.
More letter types. IOS version with KMP/Swift. Multi-lingual support. More letter types. Screenshots Screenshots Conclusion Conclusion LettersLens demonstrates how small, focused AI tools can make everyday tasks, such as opening a letter, less stressful and more actionable. Try it Try it Code: LetterLens GitHub Repo


Demo video: YouTube Dem Code: LetterLens GitHub Repo Code: LetterLens GitHub Repo Code: LetterLens GitHub Repo LetterLens GitHub Repo Demo video: YouTube Dem Demo video: YouTube Dem Demo video: YouTube Dem YouTube Dem

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

This post provides insights into new product. 

GitHub

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Building LetterLens: An OCR-Powered Android App With Kotlin + ML Kit, and Ktor

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Can AI Save Centuries of Kurdish History?

Did You Know the First Kurdish Press Was Carried by Mule?

Why Historical Documents Are Hard to Digitize

Can AI Finally Crack Ottoman Text Recognition?

Advances in OCR for Historical Chinese, Japanese, Coptic, and Greek Texts

Can AI Save Centuries of Kurdish History?

Did You Know the First Kurdish Press Was Carried by Mule?

Why Historical Documents Are Hard to Digitize

Can AI Finally Crack Ottoman Text Recognition?

Advances in OCR for Historical Chinese, Japanese, Coptic, and Greek Texts

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps