Skip to content
Rust & Native

Heavy Parsing with Serde: Offloading JSON and CSV Crunching to Rust

By David Cruz Anaya, Senior Mobile & Applied AI Engineer (Android, Kotlin Multiplatform, product AI)

Updated 2 October 202611 min read
AndroidRustNDKSerdeJSONPerformance
Heavy Parsing with Serde: Offloading JSON and CSV Crunching to Rust
Share:

Rust + NDK on Android, Part 11 of 15.This series teaches Android developers how to ship Rust in production apps, no prior Rust required. Hub: Rust on Android: How to Use It, How to Debug It, and Why It's Worth It. Previous: Low-Latency Audio DSP with Rust: AAudio and Oboe from a Rust Core. Next: Cryptography with Rust on Android: ring and RustCrypto Instead of Rolling Your Own.

Parsing JSON in Rust on Android pays off under one condition: the parsed objects never cross back into Kotlin. Parse a multi-megabyte export with serde, aggregate it on the Rust side, and return one small summary record, and the boundary costs you a single struct. Parse it in Rust and then marshal fifty thousand objects back into Kotlin, and you have paid for serde plus the expensive half of kotlinx.serialization anyway: allocating the object graph on the JVM heap.

This part builds the series’ rust/parser crate around that rule: serde derive structs for a transactions export, streaming CSV and NDJSON aggregation that holds one record in memory at a time, and a UniFFI boundary returning a compact summary Kotlin receives as a plain data class. It also explains why there is no benchmark table yet, and how I will produce the numbers.

This assumes the toolchain from Part 1 and the UniFFI wiring from Part 3. Part 6 built the cost model for crossing the boundary; this part is its advice applied: make the chunky call.

Key Takeaways

  • Serde’s derive gives you the @Serializable experience with stricter checks: the generated code is type-checked at compile time, and a field that does not match the data is a typed runtime error naming the field, never a silent null.
  • The win condition is aggregation. Return one UniFFI record across the boundary instead of a Vec of parsed rows, the chunky-call rule from Part 6.
  • Streaming through the csv crate’s record iterator or serde_json’s StreamDeserializer keeps memory flat no matter how large the file grows.
  • For ordinary API payloads, kotlinx.serialization stays the right tool. This post ships methodology instead of invented numbers; measurements land when the companion module does.

When does it pay to parse JSON in Rust on Android?

Not for your API responses. A payload of a few kilobytes deserialized with kotlinx.serialization or Moshi parses comfortably within a frame, with zero FFI in your build. If that is your whole parsing story, close this tab and keep the build simple.

The problem class is files, not responses. A bank statement export the user imports, two seasons of cached sports data for offline mode, an analytics log your app has been appending to for months. These arrive as multi-megabyte JSON or CSV, and they hurt on the JVM in a specific way: one object allocated per row plus one per nested field, the whole graph on the heap next to the raw text, and the collection pressure landing while your UI tries to hold a frame budget of roughly 16 ms at 60 Hz (Android vitals, slow rendering).

Kotlin is not helpless here, and pretending otherwise would be selling you something. Moshi reads token by token over Okio, and kotlinx.serialization has an experimental decodeToSequence for streaming from an InputStream (kotlinx.serialization API docs). What neither can change is the allocation model: every row still becomes JVM objects for the garbage collector to trace and sweep. Serde deserializes into plain structs with no collector, and when the aggregation happens on the Rust side too, the JVM heap never sees the dataset at all. That clause is the entire trade, and it is why this post returns a summary and never a list.

One boring gate before any of it: if heavy parsing were your only native need, adding rustup, the NDK and a second language to fix one screen would be a hard sell, and I said as much in the hub post. By Part 11 the toolchain is already in your build, so the marginal cost is one more crate in the workspace.

One manifest: serde, csv, and UniFFI

The crate lives at rust/parser in the series workspace. The manifest at rust/parser/Cargo.toml:

[package]
name = "parser"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
csv = "1"
thiserror = "2"
uniffi = "0.29"

Each dependency has a job. serde with the derive feature is the code generator, the ksp step of this world, except it runs inside the normal compile and its output is type-checked like everything else (serde.rs). serde_json and csv are format front ends; serde itself is format agnostic, which is why one Transaction struct will feed both parsers below. And uniffi 0.29 is the boundary, wired into the Gradle build exactly as in Part 3.

Serde structs: @Serializable with a stricter compiler

The row type sits at the top of rust/parser/src/lib.rs, together with the scaffolding macro every UniFFI crate needs once:

use std::collections::BTreeMap;
use std::fs::File;
use std::io::BufReader;

use serde::Deserialize;

uniffi::setup_scaffolding!();

/// One row of the export. Serde fills this the way kotlinx.serialization
/// fills an @Serializable data class, with mismatches surfacing as
/// typed errors naming the field, not silent nulls.
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Transaction {
    pub id: String,
    pub posted_at: String,
    pub amount_cents: i64,
    pub currency: String,
    pub category: String,
}

If you write kotlinx.serialization, you already know this struct. #[derive(Deserialize)] is @Serializable, and #[serde(rename_all = "camelCase")] does the job of @SerialName for the whole struct at once, so posted_at binds to postedAt in the data. A missing field is a typed error naming the field and position, not an exception from somewhere inside a decoder. Nullable fields become Option<String>, and #[serde(default)] fills absent values like default parameters in a data class.

The part with no Kotlin equivalent: the same struct drives the CSV parser. The csv crate matches header names against the same serialized names, so an export whose header row reads id,postedAt,amountCents,currency,category deserializes into Transaction with no extra annotations.

Return the answer, not the objects

Now the boundary types, still in rust/parser/src/lib.rs. Two records for the result and one error enum:

#[derive(Debug, Clone, uniffi::Record)]
pub struct CategoryTotal {
    pub category: String,
    pub rows: u64,
    pub cents: i64,
}

#[derive(Debug, Clone, uniffi::Record)]
pub struct TxSummary {
    pub rows: u64,
    pub total_cents: i64,
    pub largest_cents: i64,
    pub categories: Vec<CategoryTotal>,
}

#[derive(Debug, thiserror::Error, uniffi::Error)]
pub enum ParseError {
    #[error("could not read {path}: {reason}")]
    Io { path: String, reason: String },
    #[error("bad record {row}: {reason}")]
    Malformed { row: u64, reason: String },
}

The records become Kotlin data classes, as covered in Part 3. The error enum becomes a sealed ParseException hierarchy with the variant fields as properties, the exception mapping Part 8 built by hand, generated for free.

Why a summary and not Vec<Transaction>? UniFFI moves compound values across the FFI by serializing them into a byte buffer and rebuilding them on the other side (UniFFI docs). Returning eighty thousand rows means eighty thousand records lowered in Rust and eighty thousand objects allocated in Kotlin, the exact storm we moved to Rust to avoid. The aggregation lives here instead:

#[derive(Default)]
struct Accumulator {
    rows: u64,
    total_cents: i64,
    largest_cents: i64,
    categories: BTreeMap<String, (u64, i64)>,
}

impl Accumulator {
    fn push(&mut self, tx: &Transaction) {
        self.rows += 1;
        self.total_cents += tx.amount_cents;
        self.largest_cents = self.largest_cents.max(tx.amount_cents);
        let entry = self.categories.entry(tx.category.clone()).or_insert((0, 0));
        entry.0 += 1;
        entry.1 += tx.amount_cents;
    }

    fn finish(self) -> TxSummary {
        TxSummary {
            rows: self.rows,
            total_cents: self.total_cents,
            largest_cents: self.largest_cents,
            categories: self
                .categories
                .into_iter()
                .map(|(category, (rows, cents))| CategoryTotal { category, rows, cents })
                .collect(),
        }
    }
}

/// Parse a whole JSON array already held in memory, for payloads that
/// arrived as a network body rather than a file.
#[uniffi::export]
pub fn summarize_json(payload: String) -> Result<TxSummary, ParseError> {
    let transactions: Vec<Transaction> =
        serde_json::from_str(&payload).map_err(|e| ParseError::Malformed {
            row: e.line() as u64,
            reason: e.to_string(),
        })?;
    let mut acc = Accumulator::default();
    for tx in &transactions {
        acc.push(tx);
    }
    Ok(acc.finish())
}

summarize_json is the simple case: the payload crosses once as a string, becomes a Vec on the Rust heap, and only TxSummary comes back. The string crossing is not free, Part 6 built the cost model for it, but it happens once, which is the point. For data already on disk we can skip crossing the payload entirely.

Streaming CSV and NDJSON in constant memory

For files, Kotlin sends a path and Rust opens the file itself. The CSV version streams: the iterator deserializes one record, the accumulator folds it, the record drops before the next is read. Memory stays flat whether the file holds a thousand rows or ten million. Also in rust/parser/src/lib.rs:

/// Stream a CSV export from disk. One record is alive at a time,
/// so memory use does not grow with file size.
#[uniffi::export]
pub fn summarize_csv(path: String) -> Result<TxSummary, ParseError> {
    let mut reader = csv::Reader::from_path(&path).map_err(|e| ParseError::Io {
        path: path.clone(),
        reason: e.to_string(),
    })?;
    let mut acc = Accumulator::default();
    for (i, record) in reader.deserialize::<Transaction>().enumerate() {
        let tx = record.map_err(|e| ParseError::Malformed {
            row: i as u64 + 1,
            reason: e.to_string(),
        })?;
        acc.push(&tx);
    }
    Ok(acc.finish())
}

JSON gets the same shape when the export is newline delimited, one JSON object per line. serde_json’s StreamDeserializer walks a sequence of whitespace-separated values from a reader, which covers NDJSON exactly (serde_json docs):

/// Stream a newline-delimited JSON export from disk with the same
/// constant-memory profile as the CSV path.
#[uniffi::export]
pub fn summarize_ndjson(path: String) -> Result<TxSummary, ParseError> {
    let file = File::open(&path).map_err(|e| ParseError::Io {
        path: path.clone(),
        reason: e.to_string(),
    })?;
    let stream = serde_json::Deserializer::from_reader(BufReader::new(file))
        .into_iter::<Transaction>();
    let mut acc = Accumulator::default();
    for (i, item) in stream.enumerate() {
        let tx = item.map_err(|e| ParseError::Malformed {
            row: i as u64 + 1,
            reason: e.to_string(),
        })?;
        acc.push(&tx);
    }
    Ok(acc.finish())
}

One honest limitation: a giant top-level JSON array does not stream through StreamDeserializer, because the brackets make it one value. Either take the summarize_json route and hold the Vec in Rust memory, still far cheaper than the JVM heap, or write a custom serde visitor that folds elements as they parse. If you control the exporter, emit NDJSON and the problem disappears.

The Kotlin side is one suspend function

The generated bindings expose summarizeCsv, summarizeJson and summarizeNdjson as top-level functions in the uniffi.parser package, exactly as Part 3 set up. The wrapper at app/src/main/java/dev/davthecoder/rustseries/parsing/TransactionReports.kt:

package dev.davthecoder.rustseries.parsing

import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import uniffi.parser.ParseException
import uniffi.parser.TxSummary
import uniffi.parser.summarizeCsv

sealed interface ReportState {
    data class Ready(val summary: TxSummary) : ReportState
    data class Failed(val message: String) : ReportState
}

class TransactionReports {

    suspend fun buildReport(csvPath: String): ReportState =
        withContext(Dispatchers.IO) {
            try {
                ReportState.Ready(summarizeCsv(csvPath))
            } catch (e: ParseException.Io) {
                ReportState.Failed("Could not read ${e.path}")
            } catch (e: ParseException.Malformed) {
                ReportState.Failed("Bad record at row ${e.row}: ${e.reason}")
            }
        }
}

Three notes. The exported functions block their calling thread for the length of the parse, so withContext(Dispatchers.IO) is not decoration; call this on the main thread and you have moved the jank, not removed it. The typed catches work because UniFFI turned ParseError into a sealed exception hierarchy with path, row and reason as real properties. And rows: u64 arrives as a Kotlin ULong, since UniFFI maps unsigned integers onto Kotlin’s unsigned types (UniFFI docs); use i64 if you would rather hand your UI a plain Long.

One Android wrinkle: summarizeCsv wants a real filesystem path, and a content:// URI from the Storage Access Framework is not one. Copy the picked document into cacheDir once and pass that path. The zero-copy alternative, handing Rust a detached file descriptor, is Part 6 territory; for a once-per-import parse, the copy is fine and simple.

Where the benchmark numbers will come from

This is where a chart should sit, and it stays empty on purpose. The series prints measurements, not vibes, and the rust/parser module has not shipped yet. What I can give you now is the methodology, so the numbers mean something when they land.

The harness follows the rules from Part 5: release builds with R8, a named physical mid-range device, warm-up iterations before anything is recorded, medians of repeated runs rather than best case. The dataset is generated deterministically by a script committed next to the crate and swept from a few hundred kilobytes to tens of megabytes, so anyone can regenerate the identical input.

Three configurations get measured at each size. Pure Kotlin: kotlinx.serialization parsing and aggregating on Dispatchers.Default. Rust with the payload crossing: summarizeJson receiving the whole string through UniFFI. Rust streaming from disk: summarizeCsv and summarizeNdjson receiving only a path. Peak memory rides along with elapsed time, sampled via Debug.getNativeHeapAllocatedSize() and the runtime heap counters, because the constant-memory claim deserves a graph, not an adjective.

What I expect, and will happily be corrected on by my own data: for payloads Kotlin parses within a frame or two there is nothing to win, and kotlinx.serialization keeps the job on simplicity alone. Somewhere past that the curves cross, and the streaming path should pull away as files grow, because its memory use does not scale with input. Where that crossover sits is the interesting number, and the one I refuse to guess in print.

Frequently Asked Questions

Is Rust faster than kotlinx.serialization for JSON parsing on Android?

Not automatically. For small payloads the JNI crossing plus the handoff overhead outweigh the parse itself, and kotlinx.serialization wins while keeping your build simpler. Rust earns its place on large files when parsing and aggregation both stay on the native side and only a compact summary crosses back into Kotlin.

How do I pass a large file from Kotlin to Rust without copying it?

Pass the path, not the bytes. A path string is a few dozen bytes across the boundary, and Rust opens and reads the file itself. For content:// URIs from the Storage Access Framework, either copy the document to cacheDir once or pass a detached file descriptor. Direct ByteBuffers, covered in Part 6, handle data that already lives in memory.

Can Serde parse a file larger than available memory?

Yes, when the format cooperates. The csv crate’s iterator deserializes one record at a time, and serde_json’s StreamDeserializer walks whitespace-separated values, which covers newline-delimited JSON. A single top-level JSON array cannot stream this way without a custom visitor, so prefer NDJSON for anything export-sized.

Should I return parsed objects from Rust to Kotlin through UniFFI?

Only in small numbers. UniFFI serializes every record across the FFI and rebuilds it as a Kotlin object, so returning tens of thousands of rows recreates the allocation pressure you left the JVM to escape. Return aggregates, or hold the dataset behind a Rust object and expose paging methods that return one screen of rows at a time.

The rule that travels

The crate is small and the shape is reusable. Your rows become serde structs and an accumulator folds each one the moment it parses, so the only thing that crosses back is the summary. Swap Transaction for your own row type and the rest of the file barely changes. What should never change is the discipline about where parsed data lives. The moment a full list of rows crosses the boundary, you are running two serialization frameworks to do the job of one, and paying both heaps for the privilege.

Next comes the use case where Rust’s argument is strongest. Part 12, Cryptography with Rust on Android: ring and RustCrypto Instead of Rolling Your Own, swaps hand-rolled crypto for audited crates and leaves key custody where it belongs, inside the Keystore. The series index lives in the hub post.

Share:

Comments

Loading comments…